Measuring impact, maximizing resources: A strategy for effective development

We all learn from experience. In fact, real learning is impossible without some degree of trial and error. But when IFAD and its partners invest in projects to help rural households lift themselves out of extreme poverty, mistakes can be costly in both financial and human terms. That’s why it’s crucial, from the start, to have tools at hand that effectively measure the impact of every intervention.


Herder reviews pasture rotation map with a project
evaluation officer in Mongolia. ©IFAD/Susan Beccio

Impact evaluation was in the spotlight at IFAD headquarters this week during a seminar organized by IFAD and the International Food Policy Research Institute (IFPRI). It was the second in a series of research-oriented events co-sponsored by the two organizations.

While the content of the seminar was highly technical, even non-experts could grasp the importance of its key question: How can we make the best possible use of the resources we spend on development?

A credible case for support

Speaking by videoconference from Washington, DC, Maximo Torero, Director of IFPRI’s Markets, Trades and Institutions Division, offered a detailed strategy in response to that question. Answering it, he said, means assessing not only how cost-effective programmes are, but also how well they generate economic development and reduce poverty for beneficiaries on the ground.

Only by gathering evidence-based knowledge about what works, Torero argued, can aid agencies make a credible case for more donor support.

“It’s important to identify which interventions make sense and which ones don’t, and why,” he said.

Objective and systematic
Torero’s rapid-fire explanation of impact evaluation covered a lot of ground, as did the ensuing question-and-answer period. (Watch the full seminar here.) Following are a few of the presentation’s main themes.

• Causal links. Over the past decade – and especially amidst global recession and fiscal constraints – governments and other donors have increasingly demanded proof of the impact made by development policies and programmes. To meet these demands, aid organizations must clearly identify the causal pathways between interventions and results, and systematically gauge the impact of their work.
• Before and after. Objective impact evaluation hinges on a difficult task: determining what would have happened if the project in question never existed. This requires a comparison of indicators for two separate groups – beneficiaries and non-beneficiaries – both before and after the intervention. 
• Control groups. Controlled trials are one approach to making such comparisons. The trials compare randomly selected members of a beneficiary group with their counterparts in a non-beneficiary control group. There are other methods of comparison, as well. No approach is perfect, but each can be helpful in addressing the issue of attribution – that is, finding out whether changes in the status of beneficiaries are attributable to a particular intervention.
• Outcomes vs. impacts. When choosing indicators to be used for evaluation, it’s important to recognize the distinction between the outcomes and the impacts of a development project. A rural electrification initiative, for example, might result in improved indoor air quality, as households use fewer polluting fuels for cooking, lighting and heating. In this case, better air quality is the outcome. From a human development standpoint, however, the relevant impact could be a reduction in respiratory diseases and deaths among children under five.
• Design for success. To ensure an accurate assessment and to reduce costs, the impact evaluation should be designed at the inception of a project, not imposed after the project is already operational. Optimally, interventions can be evaluated at the pilot stage and then brought to scale on the basis of a rigorous analysis of lessons learned early on.

Torero returned to this last point several times during the seminar, stressing the need for “external validity” in project design and implementation. Even with large, complex projects, he said, “you need to have some testing at the pilot level to understand which interventions have the greatest impact. We can try things and see if they’re working, and once we’re sure, scale them up.”

Evaluation is, of course, simply a means to an end. For IFAD, the ultimate goal is to make a tangible, positive impact on the lives of poor smallholder farmers in the developing world. As Torero put it: “We’re not looking for a silver bullet. We’re trying to get people out of poverty on a sustainable basis.”

Missed the seminar? Watch the recording.

PowerPoint presentation