Reflections on three years of informing decisions with data and evidence

2021-03-16 blog educates evidence-informed-decision-making informs news
Reflections on three years of informing decisions with data and evidence

This blog post first appeared on the IDInsight website; click here to view the original post.

As Chris Chibwana, IDinsight’s Africa Lead and Partner, departs IDinsight, he shares five valuable lessons on evidence-informed policy. 

Until recently, I was the Head of Africa at IDinsight, an organization I joined in October 2017 after spending seven years at USAID. When I joined the organization, I had no idea what to expect. I had not heard of IDinsight prior to applying, but I was fascinated by the organization’s mission so I took the plunge, in faith. The past three years have, by far, been among the most professionally rewarding years of my career. I have had an opportunity to work with some of the best people in international development on some of the most pressing challenges facing the world today. I never thought in a million years that a boy from a remote village in rural Southern Malawi would get the opportunity to lead a key region for one of the most transformative organizations in development. I feel fortunate to have had this opportunity to contribute, in some small way, to improving the lives of many on the continent. I am also glad to leave IDinsight Africa in a strong position.

As I reflect on my experience over the past three years, five themes stand out for me that I would like to share. Each of these themes has guided and/or impacted my work in many ways and may have broader implications for advancing evidence-informed decision making in the development sector.

  1. Collaboration among suppliers of evidence can help enhance social impact – when I worked for a funding agency, I noted – and disliked – how BINGOs (Big International NGOs) competed for funding on the back of innovations and approaches that they had developed, patented, and trademarked to give them a competitive edge. When I joined IDinsight, I couldn’t help but notice the same trend among organizations that supply evidence. Many of them are in competition with each other. This is unfortunate because our common goal is (or should be) to advance development outcomes through the use of data and evidence. Perhaps I am wrong – is it possible that some organizations are not in it for social impact, primarily? But if we are all in this for the same reason, why is there not that much collaboration between organizations doing similar work? I have seen organizations compete on methods (whose methods are superior) and for funding. At IDinsight, we made a deliberate decision to prioritize collaborations with other like-minded organizations that are helping decisionmakers use data and evidence to inform decisions. To that end, we have active collaborations with a number of think tanks on the African continent. We benefit from each other’s contextual knowledge, deep and trusted relationships with funders and decisionmakers, and the sharing of analytical tools. A strong evidence ecosystem is critical to increasing the use of data and evidence in decision-making. 
     
  2. Methods are just a means to an end – measurement should be tailored - IDinsight was founded to disrupt the research and evidence “establishment” by prioritizing the needs and practical constraints of decisionmakers in the design and execution of measurement activities. This started with pioneering the decision-focused evaluation framework. During my time with the organization, I have seen us evolve from prioritizing impact evaluations to embracing a wide range of tools, including process evaluations, monitoring, and machine learning. While IDinsight cares deeply about rigor and is obsessed with the counterfactual, we realize that not all questions – including some of the most important questions faced by policymakers – are simple enough to be answered through an experiment. In most cases, the decision any given government leader may face is far from binary – act/don’t act – and it doesn’t matter much to them if alpha=0.05. The mantra, therefore, has been to deploy the right tool at the right time. While we continue to do decision-focused evaluations, my teams have worked with the Governments of Malawi and Zambia to draw valuable insights from existing administrative data that informed refinements to social protection programs in those countries. Rather than being obsessed with tools, the whole development sector would benefit if organizations that supply evidence prioritized informing decisions over fancy methods.

 

  1. Diversity, and being locally grounded, help maximize social impact – when I joined IDinsight in 2017, the majority of our staff were non-African nationals. We took steps to increase the number of local nationals in our staff. In addition to having a better and more nuanced understanding of the context within which our partners operate, a citizen of Malawi working on our partnership with the Government of Malawi has more freedom to engage in an open discussion with our government counterpart because he has a personal stake in the outcomes the government is looking to achieve. As I leave IDinsight, 50 percent of our staff in Africa are African. Additionally, the organization is moving towards having the majority of our staff based in countries where we work. One lesson we have learned from the COVID-19 pandemic is that being locally grounded is critical to the resilience and relevance of our organization. It also shows we are respectful of our partners and hosts. We witnessed at the start of the COVID-19 pandemic numerous, often frantic, relocations of expats working in global development to their high-income countries of origin in fear that the health systems in African countries where they were deployed would be overrun by the pandemic. In some instances, as with IDinsight, team members were encouraged to go where they had strong support systems and networks in case travel restrictions were put in place that prohibited movement. But still, a flurry of relocations sent a terrible implicit message to African partners and collaborators – when stuff hits the fan, you are on your own. Your lives do not matter as much as ours. You are used to suffering and death, after all. That same message is being reinforced a year into the pandemic as those “visiting expats” or “development tourists”, as Chris Blattman calls them, look to return to “the field” – to the same countries with weak health systems. This time, some will only return after they have received COVID-19 vaccines in their rich countries. Relocation policies and return-to-work frameworks are now a common thing across the development sector. While they were gone, however, their local partners continued to work to try and save lives. The returning expats will attempt to make themselves relevant, until a new and more violent wave of COVID-19 cases comes around. We were fortunate that the majority of our IDinsight Africa staff stayed on the continent – most of them because this is their home – and continued to help our partners at a time when they most needed data and evidence. When the next pandemic breaks out, I know IDinsight will be in an even better position to support our local partners.

 

  1. As an amplifier organization, humility is essential – One of the values that guides IDinsight’s work, and the one that attracted me the most to the organization, is humility. The founding partners resolved to include humility as an organizational value upon realizing that we often operate in someone else’s home, community, office or country. We recognize the limits of our personal knowledge, especially compared to local and lived experience, and believe that having multiple perspectives is highly valuable. We seek to raise the profile of the organizations with whom we work above and beyond our own. It has been incredibly refreshing to work with a group of people who embody this value deeply. Because our staff are humble, they are viewed as non-threatening by our partners. I have seen firsthand how our humility has helped build trusted relationships with government partners in Zambia, Malawi, and Ghana. As a result, our staff, despite many being relatively new in their careers, have had access to some of the most senior leaders in the development sector. And because we seek to elevate the profiles of our partner organizations above our own, we often provide evidence to our partners without demanding public acknowledgment. Such humility has also meant that we have traditionally shied away from telling stories of social impact – something the organization has only recently started to get comfortable with because it sees that sharing these lessons can be another avenue for impact.

 

  1. Partnering with governments offers the biggest path to social impact but requires patience – We decided in 2017 to prioritize long term partnerships with African governments. Since then, we have partnered with governments in Zambia, Malawi, Ghana, Kenya, Ethiopia, and Morocco to help them use data and evidence for decision-making. These partnerships are open-ended engagements in which IDinsight teams work closely with the government client (often in an embedded capacity) to answer important questions as they arise using a broad methodological toolkit. Some of the key elements of these relationships have been developing policymakers’ capacity to demand and understand data and evidence, and supporting counterpart M&E teams to supply that evidence. By doing these things, we hope to help begin addressing some of the systemic causes of underperformance within governments on the continent. Our embedded teams have contributed to improving the design and implementation of several large-scale government programs. We are still learning from this approach, including that informing government decisions takes time and that building capacity is hard work. Progress can be slow, but we are learning to be comfortable with that. Are there one-off evaluations that others have conducted on behalf of governments that informed large decisions? Yes. Did they change any of the underlying causes of underperformance within governments, including the incentives for using evidence in decision-making? No. Our government partners have expressed deep appreciation for the flexibility offered by these learning partnerships. Several government officials have expressed shock when I have asked them how our IDinsight teams can be helpful. In their experience, researchers often have their own agendas. IDinsight teams have none.

I am moving on from IDinsight to join the Alliance for a Green Revolution in Africa (AGRA) as Head of the Hub for Agricultural Policy Action (HAPA) – a new initiative that will provide decision-support services to African governments to implement policy actions that help raise the productivity and incomes of smallholder farmers across the continent. HAPA is designed as a rapid response mechanism that will leverage local research and analytical capacity in the 14 AGRA-supported countries to provide data and analytical tools to inform government decisions that help transform the agricultural sector. I am excited and looking forward to continuing the important work of partnering with governments on the continent to increase the use of data and evidence in decision-making. 

 

Disclaimer: The views expressed in published blog posts, as well as any errors or omissions, are the sole responsibility of the author/s and do not represent the views of the Africa Evidence Network, its secretariat, advisory or reference groups, or its funders; nor does it imply endorsement by the afore-mentioned parties.