On the Impact of ‘Impact’

Saturday 24th October, 2015

Robert Frodeman

Special Section on the Future of Research Governance

How can we best frame our thinking about the Research Excellence Framework? James Wilsdon doesn’t think it should come from some hypothetical universe where researchers happily pursue their work unimpeded by nosy outsiders (a world that never existed in any case). Rather, we need to find, and for that matter help create, a workable space between legitimate societal demands for accountability and the growing tyranny of the algorithm.

Yes, the concerns of the professorate are real. They were highlighted by a recent New York Times article about ugly work conditions at Amazon. As described by one Amazon employee, “The company is running a continual performance improvement algorithm on its staff.” The result is “Taylorism for the professional class” – the ability of companies to monitor their white-collar employee’s activities down to the last keystroke. The indignities that blue-collar workers have learned to live with (drug tests and full body scans when leaving the building) now are visited upon the professional class: the minute-by-minute monitoring of websites visited and work performed is coming to an office and  university near you.

Demands that publicly funded scientific research demonstrate its larger societal relevance have been a political commonplace for some time now. In the US, talk of ‘broader impacts’ at the National Science Foundation appeared in 1997, when the agency changed its criteria for the ex ante review of the 50,000 proposals it receives each year. The broader impacts criterion, paired with intellectual merit, was itself a response to the Clinton era initiative known as GPRA—the Government Performance and Results Act of 1993, which mandated that all federal agencies set goals, measure results, and issue reports on progress. Bush’s trickle-down theory of how high quality scientific research reached the public was no longer sufficient: politicians wanted proof of impact.

The US may have gotten an early start on the development of an accountability culture, but the US now lags in both theory and implementation of impact assessment. Sure, one can point to a steady critical drumbeat on the part of Republicans in Congress (e.g., calls for increased congressional oversight, or for the elimination of NSF’s Social Science Directorate). But the US has seen no national effort like those found in Europe an especially the UK.

This is partly the result of the peculiarities of the US political culture, i.e., its tradition of federalism. Public universities are state entities, which leaves each of the 50 states to develop its own metrics of impact – or not. The results of this can be quite brutal: at some Texas universities, for instance, professors are now ranked on a plus/minus dollar scale that takes account of their salary against the numbers of students they teach and their research grants. But it also leaves more room for experimentation. Moreover, it underscores a point that I and my colleagues have tried to make to our brethren for a decade now: if we do not get involved in the impact agenda it will be imposed on us from above.

In the spirit of Wilsdon’s comments, then, I would like to add one additional point. In the pell-mell pursuit of impact we have neglected to do some first order thinking on what precisely we mean by the term. Underlying our accountability culture’s focus on increasing impact is a simple assumption: impact = good, great impact = better. It is time that we stand back and review the concept. For once considered, the pursuit of impact raises as many problems as it seems to solve.

It is time for an epistemology and an ethics of impact. This point can be framed in a number of ways. To begin with, we must raise the question of harmful impacts—what are sometime called grimpacts. A moment’s reflection is enough to show the vacuity of the idea that impacts are always beneficial. Take the case of the natural environment: it is clear that in any number of cases (climate change, the loss of biodiversity) humanity is having both too many and too severe of impacts. In the future, progress in the environmental realm will often consist of lessening, eliminating, or even reversing our impact. This raises the possibility of pursuing the goal of what might be called negative impact, where the anticipated impacts of research consist of removing previous impacts.

Moreover, environmental examples like these highlight the implicit monism underlying our talk of impact. Discussions of impact have assumed that the plurality of possible impacts (economic, social, environmental, and cultural; see Donovan’s ‘quadruple bottom line’) all somehow end up pointing in the same positive direction. There is little discussion of the fact that a ‘positive’ economic impact may at the very same time be a ‘negative’ social or environmental impact. So, for instance, continued economic development often comes at a cost to the environment; or the development of driverless cars can threaten the livelihood of the 6 million people in the US who drive vehicles for a living.

There is also the question of what should constitute ‘impact’ in the humanities. Perhaps the humanities should not be seen as being about ‘impact’ at all, but rather something more like influence, affect, or nudge. This implies a shift from the Newtonian biases of ‘impact’, where something whacks into something else as in a car crash, toward more subtle quality of life indices such as personal satisfaction and belongingness. The humanities would then challenge the tacit economism of impact-talk: learning to appreciate a Picasso painting, a Keats poem, or a walk in the woods, is about expanding your imagination and sensitivity to life rather than having an impact on something or someone else.

The point is to move from simple-minded (and destructive) talk of impact to other wider, more subtle, and less presumptive notions of effect and affect. Wilsdon’s balanced approach to these questions serve as a useful guide to the choppy waters that lie ahead. 

Here's more you may be interested in: