It is a wonderful time to part of the Global Evidence Community: production of rigorous and policy-relevant research evidence is increasing, governments, NGOs, and practitioners aim to increase the use of evidence (e.g. see here and here), and a vibrant exchange of ideas is taking place across many different fora and platforms. As the year is drawing to a close, a new and different debate around evidence and its use to inform policy-making has taken off: the moral case for evidence in policy-making, introduced by Ruth Levine from the Hewlett Foundation at USAID’s Evidence Day and 3ie’s Evidence Week.
In both speeches, Ruth makes a convincing case that for too long we, the evidence community, have framed our work around technocratic and methodological contributions abstaining from outlining the moral case for evidence-informed decision-making. Making the case for evidence and making the case for social justice, one could get the impression, were two different agendas. Ruth urges the evidence community to move beyond outlining the methodological and instrumental value of investing in evidence and its use and to make a case for how evidence can “serve what really matters: justice, truth, and a better future”. This echoes similar calls for an integration of the evidence agenda with concerns for social justice and empowerment, for example expressed here by Ruth Stewart and at the Global Evidence Summit by Sarah Miller, here.
Ruth’s call also comes in timely with conversations that we are having repeatedly at the Africa Centre for Evidence in Johannesburg on how and why we think that the production of evidence that is useful and used can help decrease poverty and inequality in our region. For me personally, this renewed focus on questions of ‘why evidence’ brought back stories from my own evidence journey and how I got convinced that researching and supporting the use of evidence by decision-makers is probably the best way to spend my time if I want to contribute to socio-economic development in South Africa. So, here’s three brief stories from my personal evidence journey:
#1 Learnig how to spel—or not
Growing up in Germany, in particular West Germany, there’s a good chance that you are being taught how to spell and write through an obscure approach called Lesen durch Schreiben (Reading through Writing) developed by Juergen Reichen . I will spare you the details, but in a nutshell, Reichen’s approach holds that children should understand words by how they are pronounced and through a visual guide for each letter (Anlautabelle). I have lots of lovely memories as a child figuring out words by combining lions, elephants, and airplanes; what I don’t have 20 years later is the ability to write this text in either English or German without MS Word’s spell-checker. Spelling remains a mystery to me, unlike my sister who went to the same school a year later, but had a different teacher not following Reichen.
What I also did not know was that the vast majority of research studies, including meta-analyses for years has consistently shown how harmful Lesen durch Schreiben is for children, in particular children whose mother tongue is not German. The evidence-base is so clear that a number of provinces in Germany have banned Reichen’s practice from schools in the last 3 years. Alas, for me, this is coming too late. My primary school teacher, a lovely person, was free to use on us children whatever pedagogy she thought was best, regardless its scientific base. I am sure our teacher had good intentions, but in essence, then and now, children in Germany’s schools are experimented on daily. That such well-intended educational experiments can go rather wrong and do direct harm to learners is something I am reminded of every time I have to use pen and paper. It is not nice being a lab rat and a pinch of informed consent would have done me (my parents) good.
#2 Sport for Development—or not
Fast forward a few years and I am the very culprit of falling for good intentions and plausible theories. For five years, I worked for an NGO made up of incredible passionate and committed people using sport as a social tool for development in the rural Eastern Cape, South Africa. I headed one of the NGO’s programmes that focussed on combining football training with HIV-education and gender empowerment. This was during the build-up to South Africa’s 2010 Football World Cup. Everybody was excited about the World Cup, South Africans love football, the Eastern Cape had (and has) massive development needs; so we went implementing a football for development programme—what could possible go wrong?
Unfortunately, quite a bit. In 2010 just after the World Cup, we learned from the programme evaluation that our social intervention had made no tangible difference in the communities we worked in. Our programme was funded by a range of local and international sources, mainly public funds, and, practically, we had very little to show for it. So, you could call this an instrumental rather than an ethical case for the use of evidence, but in retrospective what is most striking to me is how obvious it should have been to us that this public money should have been spent on something else but organising social football training sessions. When discussing South Africa’s socio-economic situation and developmental needs in conversations among ourselves as well as with our friends and colleagues across the Eastern Cape, sport never came up. Rural communities were not demanding a new football programme, they were asking for investments into education, health, social services, and infrastructure. Again, with plausible theory, good intentions, and lots of passion we had experimented on perceived beneficiaries in the process depriving the communities we worked in of at least two or three schools that could have been built with the public funds used of our football for development programme.
#3 The tyranny of evidence—Ann Oakley to the rescue
Fast forward another few years, and I am now based at a research centre advocating for and supporting the use of evidence to inform public policies and programmes in South Africa. We are all social scientists with different backgrounds and in my naïve mind, I would have thought us to have an easy run convincing fellow social scientists of the value of producing useful and used research evidence in South Africa. We also work closely with government colleagues and often use joint platforms to advocate for the use of evidence. Here’s some of the catchier responses we have gotten in the last couple of years:
- You are establishing the tyranny of evidence (education audience).
- You are co-opting scientists and citizens to support government’s neo-liberal agenda (sociology audience).
- The Evidence Agenda is a meta-political project and relies on a technocratic, linear understanding of the policy making process and on a naïve empiricist understanding of the role of evidence (political science audience)
For some time, I was genuine surprised by these responses and the often hostile undertone they carry from the otherwise rather cheerful tone of social science debates in South Africa. It also shows the urgency of the call to develop a moral case for evidence and its use as in these fora, the perceived moral high-ground is certainly strongly taken by others. Fortunately, one of social science’s own gurus (in particular the more radical and critical part of the social sciences) has written extensively and beautifully about the central role of producing and using evidence of what works in order to support social justice: welcome, Ann Oakley.
As a renowned feminist and pioneer of advancing women’s rights through policy-relevant research, Oakley possesses vast credentials in advancing social justice and gender empowerment. Fortunately for the evidence community, her book: Experiments in knowing—gender and methods in the social sciences is a tour de force for the moral case to use rigorous evidence and full of practical examples of how such use has improved lives, in particular the lives of women. Two central tenants carry through her volume:
(1) You can only effectively fight oppression and marginalisation if you have reliable data of the extent of the oppression and research on the usefulness of the means that you apply to fight oppression and marginalisation.
For example, on fighting women’s oppression Oakley comments:
‘The very charting of women’s oppression required quantification, surely: we needed figures for women’s schooling vis-à-vis that of men, the distribution and work of women in the paid and unpaid labour markets, women’s earnings, the burden of health problems and so forth, in order to say to what extent the situations of men and women were (are) structurally differentiated.’ (Experiments in Knowing, p 19).
(2) Evidence of what works (and what doesn’t) can shift the balance of power from the designers and implementers of interventions to the users of such interventions. Being able to understand what educational practices are effective and which are harmful, for example, could have helped my parents speak up against the bogus teaching method I was exposed to.
Oakley makes this point more elegantly as:
‘One very important value of well-designed experimental ways of knowing inheres in the democratic ends they serve: their capacity to interrupt the ‘normal’ power relations existing between dominant professional groups and their clients. The accumulation and wide accessibility of evidence produced by this means would bring about nothing short of a revolution in the traditional relationship between the ‘lay’ public as targets of interventions, on the one hand, and the professionals and others who practice these interventions on them, on the other.’ (Experiments in Knowing, p 312).
Of course, in practice, it is slightly more complicated than this to change power relations, but as a line of argument, it certainly goes a long way in debates to reclaim the moral high-ground for evidence and its use.
To sum up then, the debates around evidence, morals, and justice are very exciting and Ruth Levine and others have already outlined a range of approaches to support a moral case for evidence in policy-making. Here, I would add two thoughts on potential inputs into this moral case:
First, social interventions, no matter how plausible in theory and well-intended in practice, have equal potential do to harm as to do good; and not using evidence on the actual effects of these interventions is akin to experimenting on children, households, communities, societies. It is simply not ethical to intervene in someone else’s lives without making the best efforts to understand the effects that this intervention might have[1].
Second, rather than a tool for methodological domination and paradigm wars, the evidence agenda should be framed as a tool for democratising knowledge and to give people a voice and choice in what type of public policies and programmes that intervene in their lives they are subjected to.
[1] See also this great blog by Tia Palermo from UNICEF making the same argument: https://blogs.unicef.org/evidence-for-action/are-random-control-trials-bad-for-children/?utm_content=buffer7cfc9&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer