Who’s Afraid of the Teaching Excellence Framework?

Image: Daniel Ruyter

Friday 8th September, 2017

Eric Royal Lybeck

Voltaire once said of the Holy Roman Empire: it was neither holy, nor Roman, nor an empire. Similarly, the UK government’s Teaching Excellence Framework (TEF) released earlier this summer is neither about teaching, nor excellence, nor is it a framework.

Still, universities will channel all their energy into further pursuit of metrics which supposedly put ‘students at the heart of the system’, but which, in fact, turn higher education into a series of Yelp! reviews. The history of education is rife with such examples of metrics becoming the cart which drove the horses.

Most university rankings, for example, are the legacy of a statistical shorthand adopted by American educationalists at the Carnegie Corporation during the 1960s. They realised ‘teaching’ was far too complex and varied to enter into a single numerical figure. Instead, they separated institutions according to function – Research I, Research II, Associates Colleges, and so forth – and, then, for the prestigious (R) groupings, they made the following assumption: because the best researchers want to work with the best students, we need only measure the best research outputs using a citation index, double it, and let that number stand in for both teaching quality and research quality.

Thus was born our contemporary obsession with research outputs, embodied since the 1980s in the UK in the, initially non-invasive, Research Assessement Exercise (RAE), which has morphed into the all-seeing, all-knowing, all-determining Research Excellence Framework (REF). Beware any who forsake the dreaded beast!

Yet, even though the REF wastes leading academics’ time to have them re-reading papers already accepted elsewhere for publication, when the submission deadlines approached in 2014, universities moved mountains to secure the best researchers, who cashed in their 4* publications to negotiate plum jobs, many of which had no requirements for teaching or even residency. And, these metrics were meant to measure departments as institutional environments for research and innovation! How could it if the academics did not even live in the country?

These and related issues led to reports, such as the 2016 Stern review, which will add new metrics to disincentivise poor behaviours produced from past metrics which were set-up to fix prior measurement issues, all which have resulted in an excessive audit infrastructure which could just as easily be pulled off the THE or QS billboard charts. All are measuring ‘research’ at the end of the day – which is why over time, the status of teaching has been steadily diminished within the university at large. Enter the TEF!

Like the first RAE in 1986, the TEF was conducted rather non-invasively, using readily accessible National Student Survey (NSS), Destination of Leavers of Higher Education (DLHE) and Higher Education Statistics Agency (HESA) statistics. And, predictably, results drawn from such terrible datasets appear to bear no resemblance whatsoever to what we know about the quality of teaching across the sector. Nearly all London universities did poorly, only 8 out of 24 Russell Group universities were awarded Gold medals at the teaching Olympics. Other big winners required extensive google searches to determine whether these ‘top’ universities were indeed real.

Regardless, even before the first results were released, the gaming of the system began. Since the DLHE scores measured employment after a bachelor’s degree – and enrolment in a postgraduate course counted as ‘employed’ – universities began rolling out the marketing materials for new Masters degrees. For a limited time only! Come one, come all! 10% discount on all MA degrees!

Slowly but surely, the TEF, designed to restore balance to a system which overemphasized research (due to a statistical assumption gone mad) will undoubtedly become more complex, more invasive, more instrumentalised, gamed, rigged and prodded until we are no longer talking about teaching, but about DHLE outputs, individual questions on the NSS, enrolment of MA students on cobbled-together courses, inventing internships. Our behaviour will change to accommodate measurements, all of which were supposed to be measuring the quality of work we academics are supposed to be in charge of assessing ourselves. Couldn’t we just do our jobs, and they measure whatever they measure and leave us out it?

For this is the ultimate issue at the heart of the TEF, which has little to do with teaching, and everything to do with who defines the ‘idea of the university’. Historically, academia was a guild established to preserve the quality of degrees, especially the degrees which conferred authority to teach at another university. This used to be simply a bachelors’ degree, but is now a PhD. Indeed, we already have mechanisms to assess the quality of academics’ teaching and research: it is called a ‘job interview’.

Beginning in the mid-19th century, however, external influences, particularly politicians and religious organizations wishing to break-up universities’ monopoly over access to the ‘establishment’ classes, began interfering with universities’ historical privileges. In the 20th century, the otherwise admirable expansion of enrolment to wider and wider populations, resulted in a mass university system, which was again expanded as we entered the 21st century to compete in the global knowledge economy. This has led to more and more external interest groups other than academics getting involved in higher education. Over the course of this transformation, academics – interested in pursuing knowledge for its own sake, but willing to lecture here and there to interested neophytes – deferred the coordination functions of the university to administrators. Eventually, these administrators took over the whole operation.

The REF and the TEF and the range of acronyms and business-speak we confront daily represents the administrators’ idea of the university. This is how they coordinate the range of demands, from students, from governments, from businesses, from philanthropies, from activists, and from ornery academics themselves. But, this is only one among many ways of looking at the university and what it is for. Indeed, the metrics and measures and the reduction of complex information into 3 or 4 buzzwords on a powerpoint slide is but one technique within schools of management, which are, lest we forget, also part of the modern university (and pay a lot of our bills).

But, this managerial idea of the university takes over only when academics themselves defer entirely to this administrative class – when we allow the administration to represent us in parliamentary committees or sector-wide associations like Universities UK. Or even when we expect the University and College Union (UCU) or the National Union of Students (NUS) to work for us because ‘we’re too busy’ (as if anyone in modern society isn’t too busy). Both the UCU and NSU inadvertently reproduce the notion that academics work for the administrators rather than the other way around. And, it was remarkable the way in which the first substantive academic opposition to the TEF emerged within the House of Lords – not because this was an autonomous elite body – but because it was the first time actually working academics, in the form of Baronesses and Lords, asked the TEF organizers: ‘What exactly are you trying to do here? Measure teaching? Raise fees? What do you mean by “excellence”’? – at which point the managers’ powerpoint presentations exploded.

So, I would suggest the history of higher education provides a fairly simply lesson for any academic working in universities today: assert clearly what your idea of the university is. Not through the UCU or NUS. Rather, through a body representing the university guild, that is: a professional association of academics, like the Royal College of Surgeons or the Inns of Court. There we could reassert our primary responsibility: maintaining the quality of degrees. In other words, teaching excellence, and research excellence in support of good teaching. All work and energy should accordingly be channelled in pursuit of these ends. It is quite clear any effort in the direction of accommodation of the TEF contributes nothing to those ends and reflects someone else’s idea of what a university education is for. I see no reason why academics should give it another thought.

Eric Royal Lybeck is a Lecturer in Sociology at the University of Exeter. He is an historical sociologist whose research focuses on the evolution of the modern research university and its changing role in society since the conferral of the first PhD in Germany in 1810. He tweets at @EricRoyalLybeck.

Here's more you may be interested in: