The Doughnut is not Enough
But it can still teach us valuable lessons about reforming research evaluation
My favourite line from The Simpsons comes when Homer brings a runaway monorail train to a halt using a makeshift anchor. Looking back from the driver’s cab he sees the anchor embedded in a giant doughnut on the roof of a trackside restaurant and sighs admiringly: “Donuts. Is there anything they can’t do?”
‘Anything’ might be a bit of a stretch in the world beyond Springfield, but you may not be aware that the humble doughnut1 now also lists economic analysis in its catalogue of achievement. “Doughnut economics” – which have nothing to do with the business of retailing sugary snacks – is an attempt to bring a richer and more sophisticated approach to the problem of characterising national prosperity than Gross Domestic Product (GDP), which is tracked by governments chasing economic growth.
GDP is a number that many have heard of but few understand. It is calculated as the sum total of consumer spending, government and business spending and investment, and the net value of imports and exports, and is commonly used as a measure of national economic activity – or to make international comparisons. Though a well-established component of economic and political dogma, GDP has long been criticised for not counting many important aspects or consequences of economic activity, such as environmental pollution, unpaid work, population health or education, income inequality, or standard of living. Robert F Kennedy Sr observed wryly that the GDP “measures everything […] except that which makes life worthwhile.”
Doughnut economics is far from the first attempt to fill the holes that have been poked in GDP, but it is one of the most wide-ranging, having been refined and extended late last year. The purpose of the doughnut metaphor, in the words of the authors, Andrew Fanning and Kate Raworth from the Doughnut Economics Action Lab at Oxford University in the UK, is to provide “a concise visual assessment of progress towards the goal of meeting the needs of all people within the means of the living planet.” The inner ring represents a social foundation, comprising 22 social indicators (e.g. access to food, water, sanitation, energy, health, education, housing; and experience of equality, social cohesion, justice and political freedom), while the outer ring denotes an ecological ceiling made up of 13 indicators that report on the health of the planet (e.g. pollution, climate change, ozone layer depletion, land conservation and biodiversity loss). The current state of the world yields a jagged, complex doughnut that makes for grim viewing. Although world’s GDP has more than doubled in the past 20 years, improvements in social foundations across the globe are falling short, while on most measures ecological harms are, like Homer’s monorail, running out of control.
The picture may be disheartening but at least it’s a picture and not a single number. The analytical value in doughnut economics lies in its multi-dimensionality, its attempt to capture the good things that money can buy and the planetary costs of the economic activity through which that money is made. More simply put, the value in doughnut economics lies in its values.
The development of doughnut economics mirrors initiatives in research assessment reform that have tried to bring a more multi-dimensional and values-based perspective to modes of research evaluation. This is the motivating idea behind initiatives such as the Declaration on Research Assessment (DORA), the Leiden Manifesto, and the Coalition to Advance Research Assessment (CoARA) amongst others. In the UK, the quest for more holistic evaluations of the research ecosystem has prompted calls to reform the Research Excellence Framework (REF)2. In particular, in recent years, attention has been drawn to the importance of monitoring research culture as part and parcel of more systemic assessments of research activity. This led to proposals, on which I was a co-author, to expand and upweight the Environment component of university REF submissions to encompass People, Culture and Environment (PCE) (alongside their selections of their best research outputs and impacts).
Subsequent pilot work to develop and test indicators to support structured narrative PCE statements with which to evaluate institutions and departments may not have produced a doughnut, but it did identify different dimensions of research culture that the REF might track to enhance the overall performance of the UK’s research ecosystem.
These were grouped into five main requirements (each associated with a proposed set of indicators): having a strategy to enhance and sustain a healthy research culture and environment; taking full responsibility for research integrity; fostering connectivity to facilitate research collaborations between disciplines, researchers and research users; using inclusivity to harness the broadest and deepest talent pools; and committing to the development of the skills and careers paths of all staff.
Categorising research culture in this way makes sense3, but the category definitions (which I have abbreviated considerably) are hardly succinct, and the resulting proliferation of PCE indicators was one of the key issues highlighted in the pilot report’s call for “clearer guidance, streamlined processes and frameworks” before they could be rolled out for REF 2029.
The complexity in defining and evaluating what constitutes a healthy research culture was likely one of the factors that led the funding bodies that run the REF to reduce the scale of their ambition for the forthcoming exercise, though no doubt opposition from research intensives, fear of stoking culture wars, and intense financial pressures on the UK university sector also played a part. Accordingly, the REF PCE component has been rebadged as Strategy, People and Research Environment (SPRE) and its contribution to institutions’ REF scores reduced from 25% to 20%4. While the guidance on preparing SPRE statements will incorporate some of the lessons and tools from the PCE Pilot, it is not yet clear how. Further guidance is due to be published “as soon as possible in 2026”.
Though an advocate of incorporating assessments of research culture in the REF (and other such exercises) as a way of incentivising healthy governance and behaviours, I still worry about the bureaucracy involved because it inevitably demands that we measure more things, that we track more indicators. In mitigation, we suggested in the Harnessing the Metric Tide report that the data gathered to allow monitoring of research culture and environment should be intrinsically valuable to institutions participating in the REF (and that the reforms be implemented step-wise over two REF cycles). But as the pilot has shown, it is difficult to draw clear boundaries around the most valuable reporting requirements.
Alongside these bureaucratic concerns, one of the more potent criticisms of the REF reform proposals is that evaluating culture and environment is looking at research inputs rather than the research outputs or outcomes that are claimed to be truer measures of success. There is some merit in this argument, given the temptation to resort to reporting things that are easy to list or count, like EDI policies or staff development courses, but ultimately the reasoning is unsound. For one thing, there are measurable outcomes from efforts to promote and sustain healthy working environments (e.g. staff demographic changes, skills development, staff satisfaction). But the more critical weakness is that to insist on outputs and impacts as the only measure of research excellence is to miss the opportunity to learn about the efficiency and health of the system. It also gives licence to an ends-justifies-the-means culture that more than a decade of work has shown to be harmful to people, and to choose to disregard the resulting detriment to quality and productivity. There is no escaping the fact that exercises like the REF have a profound impact on research culture whether they do so intentionally or not. Better therefore to choose to deploy the tool positively.
However, monitoring only matters if it changes outcomes. We are still waiting to see how the re-configured SPRE component of the REF will attempt to capture and reward positive and productive changes in the working cultures and environments of UK researchers. The important thing is, like Homer, to have a go. To experiment, to fail, to learn, and to go again.
Doughnut economics faces similar challenges. Its evidence base and theoretical under-pinnings have been criticised, but it has nevertheless fed into the United Nations ‘Beyond GDP’ project which reported earlier this year. The report proposes a dashboard rather than a doughnut, though the indicators identified as the UN’s multi-dimensional replacements for the GDP metric overlap extensively with the socioeconomic and ecological issues prioritised by Fanning and Raworth.
The authors of the UN report have grappled with some of the practicalities, limiting the scope of the dashboard of indicators to facilitate implementation and focusing on outcomes rather than inputs (or ‘enablers’) to enhance interpretability – a more explicit solution to the outputs-vs-inputs debate that has dogged REF reforms. Their work was lauded by UN Secretary-General António Guterrez for making “concrete recommendations for complementary indicators that measure what matters most to people and planet”. As with the efforts to enhance the REF by including richer descriptors of the activities that foster research quality and productivity, we don’t yet know if the dashboard will ever be properly implemented. That task has been batted to the governments of Spain and Guyana and seems likely to prove at least as challenging as arresting a runaway train, but the job is no less urgent.
Real doughnuts are a ready-made answer to the problem of mid-morning hunger pangs, so it is perhaps ironic that doughnut economics aims to transcend our usual craving for quick and easy solutions. But if we really want to understand the complex worlds of our national economies and their research systems, simplistic answers will not suffice. I doubt that we are capable of full understanding but efforts at improvement, even as we continue to argue about priorities and practicalities, will surely be worthwhile.
Critical to success will be harnessing the levers of power. Neither reports or doughnuts are enough on their own, however persuasively they might highlight issues of widespread concern or map out shared values and moral purpose. The journey still has to be undertaken. Getting nation states to move beyond the GDP will take courageous and visionary leadership, which seems to be in short supply.
Leveraging the power of the REF5, which has a big influence on block grant allocations to UK institutions, should have more traction notwithstanding the trimming of the ambition of the original proposals and the likely obstacles still ahead. It will also take leadership, and participation, at all levels within the system, and hard work, especially during a period of transition. Our aim should be to get to a place where proper attention to culture, behaviour and values becomes ‘business as usual’. RoRI is already thinking hard about these issues with its international partners, most notably in this recent paper from Cameron Neylon and this week’s Augmented Assessment Summit in Cambridge (part of our AGORRA project). The most effective tools and most important priorities will always be disputed, but that should not deter us from getting down to work. I’ll bring doughnuts.
I hope Homer will forgive me for reverting to the spelling used on this side of the Atlantic
Other systems of national research evaluation are available, as discussed recently in a recent post on the RoRI Substack.
It is important nevertheless to point out that, internationally, the terms culture and environment mean different things to different people.
This is still higher than the 15% weighting given to REF Environment statements in REF 2021.
To many UK readers, the REF may seem an unpopular vehicle for reform given that it has bloated in cost and complexity over the years, but it is not widely appreciated that as a research funding mechanism it is relatively cheap to administer, with total costs of around 3.5% of the funds disbursed, compared to around 13% for applicant-led research funding.