Los proyectos empiezan, se desarrollan y terminan, lo importante es lo que queda al final

Por Paolo Silveri

Han pasado 10 años desde el inicio del Proyecto Uruguay Rural: los primeros
cinco caracterizados por una implementación lenta e incierta, mientras
que en los otros cinco el Proyecto supo dar una vuelta de 180 grados,
logrando un largo período de acción económica y social muy intensa. El
PUR, en un contexto político favorable, supo poner en la atención de las
políticas y de las instituciones públicas el tema de la pobreza y del desarrollo
rural, cuando más de una voz en el país minimizaba la existencia
misma de un problema de pobreza en el campo. El Proyecto propuso y
aplicó herramientas y metodologías innovadoras, considerando como eje
central de su labor las personas, con sus aspiraciones, sus estrategias,
sus recursos humanos, sus organizaciones, su capacidad productiva. Y ha
tomado ese camino con mucha determinación y claridad, tanto estratégica
como operativa.

Los proyectos empiezan, se desarrollan y terminan. Lo importante es lo que
queda al final y, en el caso del PUR, queda mucho. Los resultados y las metas
físicas están en este y en otros informes, además de ser visibles en
los ingresos y en la capitalización de las fincas y casas de los pequeños
productores y pobres rurales que han participado en el Proyecto. Algunos
intangibles, como el apoyo para la creación de la Dirección General de Desarrollo
Rural en el MGAP y las Mesas de Desarrollo Rural en los departamentos
de todo el país, así como las muchas organizaciones locales de productores
fortalecidas, también están a la vista y, para el Fondo Internacional de
Desarrollo Agrícola (FIDA), son logros extremadamente relevantes, pues representan
un espacio totalmente novedoso en el panorama institucional
para el sector y una plataforma que permitirá continuar en el proceso de
desarrollo rural y de políticas públicas que lo sostengan en el tiempo.

La implementación del PUR ha sido marcada por una capacidad novedosa
y fructífera de obtener lo mejor posible del encuentro entre una agencia
financiera internacional (el FIDA) y la institucionalidad nacional (el MGAP y
el Ministerio de Economía); entre la intervención del Estado y las expectativas
de la sociedad civil; y del encuentro entre las oportunidades del
mercado, la tecnología, las inversiones con las personas, que han sido el
norte permanente en la orientación de las actividades del PUR.
El equipo del PUR supo hablar y escuchar a los hombres y a las mujeres
del campo. Ha propuesto, ha polemizado y ha mediado, siempre buscando
una solución que acompañara las aspiraciones de los pequeños productores
y los pobres rurales. Y en la mayoría de los casos lo ha logrado. Las
visitas de campo del FIDA han relevado con frecuencia el testimonio directo
de pequeños productores y productoras que, antes de mencionar las
mejoras en la productividad y en las condiciones de vida en sus familias
y comunidades, afirmaban que el PUR les había "escuchado y devuelto una
esperanza".

Un equipo joven, con la mayoría de los funcionarios y técnicos con menos
de 30 años, una combinación de agrónomos con especialistas de las Ciencias
Sociales, el compromiso personal, las ganas de analizar, de aprender
y de discutir sin cansancio, ha generado un ambiente de trabajo y un tejido
de relaciones con los pequeños productores y pobres rurales basados en
el respeto mutuo y en la profesionalidad con compromiso. Cuando se cometieron
errores, se debieron a exceso de entusiasmo, de ganas de hacer,
de confianza y optimismo, y al principio a la inexperiencia; pero nunca a la
falta de compromiso, de dedicación o de interés. Existe ahora una masa
critica de profesionales jóvenes que han acumulado una importante experiencia
en el desarrollo rural, que esperamos el país sepa aprovechar.
Los logros físicos han sido fundamentales para permitir el logro de los
resultados intangibles, y viceversa. Sin embargo, quizás el mayor aporte
del PUR al Uruguay ha sido definir un camino para que las políticas públicas
logren apoyar y acompañar eficazmente las estrategias y las aspiraciones
de los pequeños productores y pobres rurales, para ayudarles a salir ellos
mismos de sus condiciones de pobreza.

Desde el FIDA, reconocemos que, con ganas, profesionalidad y compromiso,
el PUR ha logrado resultados importantes y sostenibles. El haber podido
acompañar este proceso ha sido para mí un privilegio y una verdadera
satisfacción profesional y personal. Espero que esta experiencia sirva de
impulso para que el país siga creciendo en la identificación, formulación y
actuación de políticas públicas con los hombres y las mujeres del campo,
y no solo para ellos.

Paolo Silveri
Gerente de Programas / División de América Latina y el Caribe.
Fondo Internacional de Desarrollo Agrícola




Leer más

Proyecto Uruguay Rural – Informe de cierre Evaluación de resultados y percepciones de los involucrados español

Compartir

Measuring impact in rural Vietnam: IFAD tests a new approach

Children walk to school in Hoang Su Phi, Viet Nam, in 2002.
Since then, the country has halved poverty overall.

Viet Nam has come a long way since its days as a net importer of food in the early 1980s. Today, following decades of robust growth, it is the world’s second largest exporter of rice. In the past 10 years alone, according to World Bank statistics, it has also halved the proportion of its population living in poverty. Yet inequities persist, especially in remote rural regions where many of the country’s ethnic minority groups live. IFAD continues to support five projects in these areas, working with smallholder farmers to reduce poverty through sustainable agricultural development.

This past week, staff and partners from the IFAD country office in Hanoi joined colleagues at headquarters in Rome by video link. They convened to discuss a newly expanded approach to monitoring and evaluating the impact of IFAD-supported projects in Viet Nam. It was the third in a series of research seminars co-sponsored by IFAD and the International Food Policy Research Institute (IFPRI), which has collaborated, as well, on developing this new approach.

The discussion was not without controversy. Still, there was a tangible sense of urgency, on all sides, about stepping up IFAD’s ability to accurately measure results on the ground – not only in Viet Nam but worldwide.

Ambitious target
Thomas Elhaut, IFAD’s Director of Statistics and Studies for Development, moderated the seminar. He began by placing the Viet Nam experience in a broader context: namely, the ambitious target set during the latest replenishment of funding from IFAD member states. The target calls for lifting 80 million rural people out of poverty by 2015.

Without improved evaluation, Elhaut suggested, IFAD will not have the evidence base to design and implement projects that can help reduce poverty on such as large scale. Nor, in the end, will it have the data to credibly report on progress toward the goal of 80 million.

From Hanoi, Atsuko Toda of IFAD and Nicholas Minot of IFPRI went on to describe the approach being tested in Viet Nam and why, in their view, it may hold promise for other countries. The methodology builds upon IFAD’s existing results and impact management system, or RIMS, which has been in use since 2003. By expanding the standardized RIMS household survey, this approach – dubbed “RIMS-plus” – aims to capture more detailed data that is relevant to specific project aims. In another innovation, it uses a control group of households, located outside the target area, to better assess which results are attributable to a project.

Additional indicators
Toda noted that the expanded questionnaire goes beyond the main objectives of the RIMS survey, which were to measure household assets and child nutrition. The RIMS-plus survey, she said, collects information on many additional indicators to diagnose constraints faced by farmers, including limits on access to rural markets, extension services and credit.

Minot added that the designers of the expanded questionnaire have still kept it relatively brief compared to most other household surveys; as a result, it should not overburden either enumerators or respondents.

Regarding the control groups, Minot explained that they would be selected from populations who live near project areas and whose backgrounds and economic status are similar to those of project beneficiaries. “It helps us to adjust for changes in the standard of living outside the project area,” he said. For example, a drought in the same province where a project is under way could cause an overall drop in rural incomes, which would have to be factored into any analysis of the project’s impact.

A third participant in the video link, Nguyen Ngoc Ahn of the Hanoi-based Development and Policies Research Centre, outlined the cost implications of both the extended questionnaire and the use of control groups. The questionnaire increases the amount of time needed to complete each survey and the training needed by enumerators, he said. The control groups add 300 households to the standard RIMS sample of 900 for large projects.

Costs of customization
At this point, some sceptical voices in the audience at IFAD headquarters questioned whether the new methodology’s value justified its added expense. Specifically, they asked whether the control groups could truly mirror the beneficiary households, given the complexities of rural poverty – including gender dynamics. In addition, they wondered whether the benefits of customized household surveys (more project-specific information) outweighed the advantages of standardized questionnaires (lower cost and consistent data across multiple projects).

In response, Toda noted that some of the expenses in Viet Nam represented the one-time cost of researching and developing a new evaluation tool. And while Minot acknowledged that the control groups could not match every household parameter, he maintained that they would provide valuable comparisons with project beneficiaries nonetheless.

On the question of standardized vs. customized surveys, Minot proposed a compromise: “We can use standardized modules but let each country decide which modules are most relevant for them,” he said. Such an arrangement would allow for a degree of customization without adding excessively to costs, he said.

As the seminar wrapped up, Thomas Elhaut praised the presenters in Hanoi for advancing IFAD’s thinking on the design and analysis of agricultural projects, even as various questions remain to be resolved. To some extent, those questions will be answered as the RIMS-plus experiment proceeds in the uplands and deltas of rural Viet Nam, where smallholder farmers need the tools to build a sustainable future.

Watch the recorded webcast of the IFAD-IFPRI seminar:

The growing role of "drones" in agriculture

For those who follow my work on the topics of remittances, migration and development a warning; for once you will hear nothing about any of these topics from me. This post does however, combine my work at the International Fund for Agricultural Development (IFAD), with another fascination of mine: designing, testing and flying large remote control aircraft.

While these aircraft may seem like large toys to the uninitiated, technology has advanced to the point where autopilot systems, GPS tracking, small cameras and arrays of transmitters can allow you to fly as far as 50 kilomiters from the point of launch, all while following the flight on a screen as if you were piloting a real aircraft. The vast number of sensors and cameras carried by what are called unmanned areal vehicles (UAVs) can be used for many purposes. Most of you will be all too familiar with the military and intelligence applications of their larger brothers such as Global Hawk and Predator, but there are also a host of civilian applications for the technology. UAVs for instance, can make detailed 3D maps of fields, forests, moutains, rivers and lakes and can play important roles in everything from detecting forest fires to monitoring crop growth. Today I discovered a BBC article that is a great introduction to some of the new applications of this technology. It doesn't take a great deal of imagination to see that UAVs can also offer a cost-effective solution for monitoring vast areas of land in developing countries.

The article provides a good introduction to what is possible, and the fact that the BBC is reporting on the topic highlights just how serious the contribution of these "model aircraft" is being taken!

---

Spying on Europe’s farms with satellites and drones

Source: BBC
By: Laurence Peter
Farmers who claim more EU subsidies than they should, or who break Common Agricultural Policy rules, are now more likely to be caught out by a camera in the sky than an inspector calling with a clipboard. How do they feel about being watched from above?

Imagine a perfect walk in the country, a few years from now - tranquillity, clean air, birdsong in the trees and hedgerows, growing crops swaying in the breeze.

Suddenly a model plane swoops overhead.

But there is no-one around manipulating radio controls. This is not a toy, but a drone on a photographic mission.

Meanwhile, hundreds of kilometres up in space, the same patch of land is being photographed by a satellite, which clearly pinpoints individual trees and animals.
What is there to spy on here? No secret military installations, just farmland.

But Europe's farms cost taxpayers billions of euros in subsidies each year, and EU agricultural inspectors are turning to technology to improve their patchy record on preventing fraud and waste.

Satellites have already been in use for several years, and drones are currently undergoing trials.

Scanning a farm with a satellite costs about one third as much as sending an inspector on a field visit - £115 ($180; 150 euros) rather than £310 ($490; 400 euros), says the UK's Rural Payments Agency (RPA), which is responsible for disbursing the subsidies in the UK and checking for irregularities.

"The RPA follows up only on those claims where there is some doubt about accuracy, and then only at the specific fields for which the doubt exists," the RPA says. "This saves time, lifts the burden on farmers and reduces cost to the taxpayer."

Satellites can rapidly cover a huge area in detail and quickly return to photograph it again if necessary.

In 2010, about 70% of the total required controls on farm payments in the EU were done by satellites, which photographed more than 210,000 sq km (81,000 sq miles) of land in all.

But they are not infallible. Austria does not use them, on the grounds that the shadows cast by very mountainous terrain sometimes make satellite images inaccurate.
And Scotland, unlike the rest of the UK, decided against satellites "because of the difficulty of getting enough clear weather for flyovers", a Scottish government spokeswoman told the BBC.

Many things in the countryside are constantly changing and when the satellite passes over, "the animals may be in a field or in a barn - you can't count the numbers very well", says Roland Randall, an English farmer and environmental researcher in Cambridgeshire. "When planners looked at the aerial photo records of our farm they thought we had an additional building without permission, but it was actually a haystack," he told the BBC.

The satellite checks are done partly to produce accurate maps of farms, showing clearly the areas eligible for subsidies.

But farmers these days have to keep their land in "good agricultural and environmental condition" to qualify for subsidies, so images also reveal whether the farmer is complying with the rules on hedges and ponds, say, or buffer strips around arable fields.


The monitoring is not always to spot infringements or illegal activities. Satellite images can help identify problems such as erosion. Farmers can then be asked to change their methods to prevent further damage. In the above images, overproduction and deep ploughing in clay earth meant rain washed soil away.

A farmer who breaks the rules risks losing 3% of his or her direct payment - and more if it is a repeat infringement.

There have been few prosecutions in the UK based on satellite evidence, says Ray Purdy, a senior law researcher at University College London (UCL) specialising in satellite monitoring.

One case in the UK was dropped in 2001 because a farmer proved that he had planted a linseed crop, even though the satellite image appeared to show bare earth. The sparse young plants had failed to show up against the bright reflection off chalk downland. This could be the kind of situation where a drone - an Unmanned Aerial Vehicle (UAV) - would come in useful.

Drones are best known for their role as remote-control killers in Afghanistan, but supporters see a role for smaller and much simpler drones in agricultural monitoring. They can get up close and take sharp photographs - and unlike satellites, which always look directly down, drones can get an angled view of their subject.

They are currently being tried out in vineyards in the south of France, to check that "grubbing up" of vines is done legally and ecologically.

Wine-growers get as much as 10,000 euros ($13,000; £8,300) per hectare in subsidies for digging up uncompetitive vines - a scheme to prevent new EU "wine lakes" caused by overproduction.

"There has to be 100% control, as it's a huge amount of money," says Philippe Loudjani, an agronomist at the Joint Research Centre, the European Commission's main satellite monitoring hub in Ispra, northern Italy.

"The French are testing to see if the drones need to go up to 10cm resolution - to see what accuracy is required."

Drones are also being tested in Italy, and are already in use on a small scale in Spain's Catalonia region, where authorities say their 25cm and 12.5cm resolution photos are ideal for inspecting the small landholdings with mixed crops that are typical of the Mediterranean.

The EU is hurrying to develop a "strategy for Unmanned Aircraft Systems", which would see the existing very tough restrictions on the use of civilian drones in Europe relaxed.

A discussion paper prepared for a European Commission workshop in Brussels this week, envisages their use not only in crop or farm monitoring, but also terrain cartography, goods transport, monitoring of borders, the fight against illegal immigration and drug trafficking, and intervention in natural or industrial disasters.

 "They can also be sent to deliver rescue packages to ship crews in danger at sea," it adds.

However, in the short term it's likely that UAVs will only be widely approved for use within line of sight of an operator and at a distance of no more than 500m, which limits their value for agricultural inspection.

And what about the privacy issue?

Ben Hayes of the campaign group Statewatch worries that Europe is rushing into the use of drones without sufficient public discussion.

"We would accept the argument that there are lots of things they can be useful for, but ... the questions about what is acceptable and how people feel about drones hovering over their farmland or their demonstration - these debates are not taking place," he says.

Ray Purdy of UCL surveyed 202 farmers in the UK, and 428 in Australia - where satellites are routinely used to monitor land use, especially vegetation clearance - and found that only about a quarter in Australia and a third in the UK were against satellite monitoring.

Some farmers voiced concerns about invasion of privacy, but many said remote sensing was preferable to inspectors on their land taking up their time.

A majority in the UK also agreed that the satellites would help to deter fraud.
Rob Allan, a farmer in Warwickshire, said "it's modern life really - I don't think there's anything you can do about it".

Measuring impact, maximizing resources: A strategy for effective development

We all learn from experience. In fact, real learning is impossible without some degree of trial and error. But when IFAD and its partners invest in projects to help rural households lift themselves out of extreme poverty, mistakes can be costly in both financial and human terms. That’s why it’s crucial, from the start, to have tools at hand that effectively measure the impact of every intervention.


Herder reviews pasture rotation map with a project
evaluation officer in Mongolia. ©IFAD/Susan Beccio

Impact evaluation was in the spotlight at IFAD headquarters this week during a seminar organized by IFAD and the International Food Policy Research Institute (IFPRI). It was the second in a series of research-oriented events co-sponsored by the two organizations.

While the content of the seminar was highly technical, even non-experts could grasp the importance of its key question: How can we make the best possible use of the resources we spend on development?

A credible case for support

Speaking by videoconference from Washington, DC, Maximo Torero, Director of IFPRI’s Markets, Trades and Institutions Division, offered a detailed strategy in response to that question. Answering it, he said, means assessing not only how cost-effective programmes are, but also how well they generate economic development and reduce poverty for beneficiaries on the ground.

Only by gathering evidence-based knowledge about what works, Torero argued, can aid agencies make a credible case for more donor support.

“It’s important to identify which interventions make sense and which ones don’t, and why,” he said.

Objective and systematic
Torero’s rapid-fire explanation of impact evaluation covered a lot of ground, as did the ensuing question-and-answer period. (Watch the full seminar here.) Following are a few of the presentation’s main themes.

• Causal links. Over the past decade – and especially amidst global recession and fiscal constraints – governments and other donors have increasingly demanded proof of the impact made by development policies and programmes. To meet these demands, aid organizations must clearly identify the causal pathways between interventions and results, and systematically gauge the impact of their work.
• Before and after. Objective impact evaluation hinges on a difficult task: determining what would have happened if the project in question never existed. This requires a comparison of indicators for two separate groups – beneficiaries and non-beneficiaries – both before and after the intervention. 
• Control groups. Controlled trials are one approach to making such comparisons. The trials compare randomly selected members of a beneficiary group with their counterparts in a non-beneficiary control group. There are other methods of comparison, as well. No approach is perfect, but each can be helpful in addressing the issue of attribution – that is, finding out whether changes in the status of beneficiaries are attributable to a particular intervention.
• Outcomes vs. impacts. When choosing indicators to be used for evaluation, it’s important to recognize the distinction between the outcomes and the impacts of a development project. A rural electrification initiative, for example, might result in improved indoor air quality, as households use fewer polluting fuels for cooking, lighting and heating. In this case, better air quality is the outcome. From a human development standpoint, however, the relevant impact could be a reduction in respiratory diseases and deaths among children under five.
• Design for success. To ensure an accurate assessment and to reduce costs, the impact evaluation should be designed at the inception of a project, not imposed after the project is already operational. Optimally, interventions can be evaluated at the pilot stage and then brought to scale on the basis of a rigorous analysis of lessons learned early on.

Torero returned to this last point several times during the seminar, stressing the need for “external validity” in project design and implementation. Even with large, complex projects, he said, “you need to have some testing at the pilot level to understand which interventions have the greatest impact. We can try things and see if they’re working, and once we’re sure, scale them up.”

Evaluation is, of course, simply a means to an end. For IFAD, the ultimate goal is to make a tangible, positive impact on the lives of poor smallholder farmers in the developing world. As Torero put it: “We’re not looking for a silver bullet. We’re trying to get people out of poverty on a sustainable basis.”

Missed the seminar? Watch the recording.

PowerPoint presentation

Action plans and indicators - the road leading towards the end of the EU Food Facility in West Africa

The M&E workshop in preparation of the Final Report for the IFAD/EU Food Facility Programme in West Africa closed today.

Over the two days of the workshop, discussions and energy levels were at their best. It was a real pleasure to facilitate a workshop of this type with such a motivated group.

The day began by the participants giving feedback on the data collection tools developed the previous day. Each team was asked to review the questionnaire and personalise it to fit the needs of the country and context. The precisions brought to the document were mainly concerning some thinking about the partners and stakeholders involved in the implementation of the Food Facility. We also reviewed some semantics one again, as the questionnaire needs to be as clear as possible and a stand- alone document for someone who has not participated in the Ouagadougou workshop where the document was conceived.

The floor was then given to ICRISAT who presented their qualitative data collection questions to the rest of the group. The ICRISAT questions necessarily needed to differ from those of the country components as the institutional arrangements for the ICRISAT grant are somewhat different.

In any case, for both the country components and ICRISAT, the questions are centered around the following themes:

- Relevance of the programme and its similarities to other food security programmes

- Farmer assessment of the trainings/knowledge and support received

- Sustainability and probability of scaling up interventions/services once the project is closed

After agreeing on the questions, the participants with the guidance of the M&E Consultant, Task Manager and Head Agronomist for the EU FF, drafted an action plan including an implementation schedule going up until the discussion and endorsement of the final report to be submitted to the EU which will take place in Abidjan at the end of March 2012.

The first step in the action plan is to recruit a local consultant by component to alleviate project teams in the collection of the qualitative data set that was agreed upon for the EU FF.

At the end of the workshop, the particpats expressed their enthusiasm regarding the methodology and approach used to tackle the preparation of the final report to the EU Food Facility. All participants recognised the lack of qualitative data in many of the reports produced and appreciate the innovative approach towards producing a final report which truly incorporates impact.

As many pointed out though, it would have been very useful to have this workshop well before January 2012. The report writing team and participants are in any case motivated and ready to achieve objectives set forth during the workshop and translated into the challenging action plan.

Indicators, indicators! They really get people talking!

Participatory M&E workshop towards the preparation of the Final Report on the EU Food Facility in West Africa

by Adriane Del Torto

Today, we opened the two day workhop on M&E for the EU Food Facility. Present at the worksessions were the M&E specialits and Production specialits from Benin, Mali, Senegal, Ghana and ICRISAT. The meetings took place in Ouagadougou, Burkina Faso at 'Eau Vive" a local NGO on 26 and 27 January 2012.

The objective of the meeting is to take stock of the efforts already made in M&E at the project level and to come up with an action plan to enable to collect qualitative data for the final report to the EU. The idea of a participatory M&E workshop came from some brainstorming between myself and some CPMs and the M&E consultant who is part of the team producing the final report as a solution to the lack of time for a mission visiting all components of the Food Facility in less than one month. 

The first day of the workshop was mainly to discuss results achieved and activities which did not take place and why. Then the group went through all of the EU indicators and discussed the relevance of many of those requested by the programme. In many cases, it was pointed out that baseline data was missing and that it would be difficult to report on all of the indicators in both the Logframe and the EU table. Each indicator was discussed and put into context to ensure proper understanding of the mechianisms.

Thi step was not planned on the original agenda, however, when considering the complexity of the programme, made of 7 distinct grant agreements, it became quite relevant to discuss requirements in order to have a somewhat harmonious report.

One we had agreed on the quantitative indicators, the rest of the day was dedicated to the qualitative ones. Adding a qualitative dimension to the final report of the EU FF was suggested to the EU during my last mission in Brussels in December 2011. My proposal was very much appreciated by the Programme Manager at the EU, who is very much looking forward to reading our results.

The lead M&E consultant and the group identified a series of themes and specific questions to ask programme stakeholders in order to collect the qualitative type data sought. These were accompanied by a list of information sources and data verification documents.

This first day, was a bit long, however, the output was quite positive. The debates between participants were lively and engaging and the colleagues present were strongly motivated by others' comments and questions. Let's  hope the ambiance at its best tomorrow as well.

Dernières lignes droites avant la clôture du PFA : Réunion Suivi-Evaluation

Le programme de la Facilité Alimentaire (PFA) de l’Union Européenne en Afrique de l’Ouest entre dans sa phase finale. Le FIDA, comme stipulé dans les accords avec l’Union Européenne, doit soumettre un rapport de fin d’activités au plus tard le 31 mars 2012.
L’objectif général de ce rapport est de présenter les principaux résultats obtenus par le Programme, les difficultés rencontrées dans la conduite des activités, les leçons et les enseignements à tirer de cette expérience, de renforcement des capacités des différents pays en matière de production et de diffusion de semences et de boutures certifiées.

Dans l’optique de la rédaction de ce rapport, une réunion sur le suivi-évaluation aura lieu à Ouagadougou, Burkina-Faso, du 25 au 27 janvier 2012.

La réunion, composée de responsables suivi-évaluation et de chefs de composantes du PFA, sera menée par Adriane Del Torto, Chargée de Programme Associée, par M. Daniel Thieba, consultant en Suivi-Evaluation et par M. Fatogoma Diarra, consultant expert agronome.

Les objectifs de l’atelier sont les suivants :
Faire le point sur l’état d’avancement dans la collecte des informations sur la mesure des indicateurs
Faire le point sur le travail restant à être réalisé (données statistiques et collecte des données qualitatives auprès des paysans)
Rappeler et identifier les indicateurs complémentaires à ceux déjà mentionnés dans les cadres logiques, pour mesurer l’impact du programme
Préciser la répartition des rôles entre les acteurs partis prenantes de la conduite de l’évaluation
Préparer les visites sur le terrain pour le mois de février 2012.

Suite à ces deux jours de réunion, un plan d’action sera adopté.

Monitoring and Evaluating the IFAD Country Programme in Ghana

by Adriane Del Torto
 
Monitorting and Evaluation (M&E) is often considered one of the weakest links in development programmes. This is also true for IFAD. The IFAD Country Programme in Ghana took this as a challenge to overcome and started to invest in M&E about two years ago. After putting an emphasis on this theme for some time now, it was finally decided that the stakeholders involved in M&E should get together to share successes and address the gaps that still exist in M&E. From 12 to 14 December, a Ghana Country Programme Learning and Exchange Workshop on M&E took place at the Alisa Hotel in Accra.

The workshop , organised by the IFAD Country Office in close collaboration with the West Africa Rural Foundation (WARF) brought together representatives from all IFAD projects, the Roots and Tuber Improvement and Marketing Project (RTIMP), Northern Rural Growth Programme (NRGP), Rural Enterprises Programme( REP II), Rural Agriculture Finance Programme( RAFIP), Northern Region Poverty Reduction Project (NORPREP), WARF, the Government of Ghana - the Ministry of Food and Agriculture (MOFA), Ministry of Finance and Economic Planning (MOFIP) and the National Development and Planning Commission (NDPC) and partners IFPRI/SAKKSS, IPA, University of Ghana, Busylab , Esoko Mobile Technology, in  an innovative and interactive workshop.

The overall objective of this workshop is to identify how to use M&E to its fullest capacities. This means using the information generated by M&;E on implementation progress as a management tool rather than an institutional requirement. The workshop helped identify outcomes, output and impact of projects while emphasising on the qualitative side of M&E which is often excluded.

The basis of the discussion were the newfound focus on M&E for the Ghana Country Programme, the recommendations from the last country programme supervision mission (which took place mid November to mid December 2011) , the current national and project M&E structures, capitalisation efforts and finally research in M&E undertaken by our implementation partners.

After each project identified the key issues raised by the supervision mission, the partners making their presentations, the workshop allowed each project to review the M&E indicators making sure they are SMART (Specific, Measurable, Achievable, Relevant, Time bound) and ensure that these indicators are the right one to measure the outcomes and impacts desired by the project. It also allowed for the identification of research needs that will improve the M&E and impact reporting

The workshop highlighted the good results in M&E in the Ghana portfolio until now, and also how much these efforts contribute daily to the improvement of the systems and the improvements still needed. Most importantly, projects have shared and exchanged on how to improve their indicators and how to address the difficulties they are facing. Some recurrent themes are the following:

- need to improve data collection at the field level by involving beneficiaries and possibly digitalising

- need to strengthen grassroots institutions
- need for data control at all levels
- need for better data management and analysis
- need for commitment of all actors
- need to fill information gaps through studies
- need to identify, undersand and maintain linkages
- need for better gender mainstreaming
- need to identify further partnerships through good planning

Each project came up with an action plan during the workshop that will be mainstreamed into the AWPB to ensure its implementation.

Most importantly, a topic that came up on many occasions is the importance of good communication and the importance of making the time to exchange on the positive outcomes achieved. In order to ensure good communication there are several Fora in Ghana to ensure exchanges between stakeholders on themes of on microfinance, rural finance and value chains. Furthermore, WARF and FIDAFRIQUE are imporant partners in Ghana but also for IFAD projects in West and Central Africa: they help will ensure that the information to be shared is well articulated and diffused on the regional platform.


Outstanding challenges and considerations for the future are how to make adequate time to engage in proper M&E activities and to ensure that the lessons learned from good M&E fit into policy dialogue.

“Happiest man in Africa…….and change is possible….”.

By Willem Bettink

I participated in the annual regional  implementation workshop of the East and Southern Africa  region. A full week immersion in Africa, to be more precise in northern Tanzania,  at the foot of mount Kilamanjaro. It was an  exciting and dynamic 4 days, not in the least thanks to “veteran” Edward Chumamoto. He facilitated -189 people- with grand style, a sense of humour and enormous skill –a true pleasure!

The project coordinators in the region decided last year to radically change the format as they felt there was not enough interaction and added value generated.  The workshop embraced the  open space methodology which generated many unexpected sessions about a variety topics from learning routes, communication and social media, gender equality and many more.

The theme of this workshop was: Managing for Impact-one of those development  concepts that appear clear at face value, but are not . As a participants said : “over these 4 days we have unpacked the concept and it has become clearer for me what it means to me in my work in our project”. 

Over the days we unpacked the managing for impact into its key elements from the perspective of a project team . One of the  results was a practical diagnostic tool referred to as the learning wheel for managing for impact. It brings together  12 elements that enable a project team to diagnose its performance.

The outcome of this diagnosis enables a team to discuss and agree on corrective actions to improve its delivery to achieve impact. One of these elements is “ continuous experimentation with new ideas and approaches as a source of innovation and performance improvement”.   

Almost all- if not all- development programmes are change management programmes. Research and our own experience  has proven  that a typical organizational change programme has a 20-30% success rate . Or if we  look at innovation and start –up businesses: only 10-15% make it through year  1 and go on to achieve sustainable profits.

If we truly believe in “managing for impact",  it implies that project teams, given the change and innovativeness of development programmes,  continuously need to perform at the top of their toes. Delivery has to be of an exceptional quality while it is conditioned by known obstacles , unplanned interferences, unexpected natural disasters and what have you.

This implies that a project team  needs to have  a strong predisposition to openly reflect upon its challenges and mistakes , harvest the learning (in particular from failed attempts ) and share  this learning with its stakeholders.

In my opinion , managing for impact is closely linked to return on investment. If we do not focus persistently on achieving impact, creating value,  we are not ensuring the pay-off to the rural poor for what they invest to engage with the changes and innovations promoted by development programmes.

A final personal note: that change is possible whether at  personal, process, technological level  I am strongly convinced off. While I was in Arusha  I witnessed the transformation of a colleague to be the happiest man in Africa.  

Whither M &E in IFAD projects? – A Modest Proposal

By Steven Schonberger

Where I come from, in Oakland, California in the US, if you want to raise the temperature in a room you just have to mention the “immaculate reception”; the moment in 1972 when the Pittsburgh Steelers football team scored a winning touchdown against the Oakland Raiders in what we Raiders fans are sure was an illegal play which eliminated them from the playoffs and a potential Super Bowl victory (similar to Diego Maradona’s “mano de Dios” against the English in the 1986 World Cup quarterfinals).  In IFAD, I have noticed that a similar catalyst of passions is the mention amongst CPMs of the Results Indicators Management System, or RIMS which can turn any meeting into an exchange of frustrations and complaints.  My sense, however, is that this is not necessarily a response to RIMS, per se, but rather to the difficulties of implementing M&E systems in general in donor financed projects.  In this case, IFAD’s RIMS is perhaps “the best of the bad”.  Fortunately, in M&E, as in the case of football, there are always more games to play and opportunities to do better if we can improve our game.  Following are a few ideas of how we might do that.

In July, we benefitted from a visit by  Rachel Glennerster, the Executive Director of the Abdul Latif Jameel Poverty Action Lab (J-PAL) at the Massachussetts Institute of Technology which has gained international renown for implementing and advocating the use of Randomized Control Trials (RCTs) to assess the impact of development activities ranging from the use and willingness to pay for improved mosquito nets to incentives to keep girls in school to the household impacts of access to microfinance.  Drawing on this experience, Rachel emphasized some key points of importance to IFAD, including:

·         The trend is that RCTs are becoming more common as the methodological standard expected by funding agencies;

·         The good newsis that improved approaches to establish control groups are broadening the situations where RCT can be applied;

·         The bad newsis that some funders are insisting on more use of RCTs than is either useful or feasible;

·         The challengeis to know when it makes sense to use rigorous, RCT-type approaches and what other types of approaches should be used based on what information is needed and what information can be realistically generated.

A key outcome of the subsequent discussion was that donors, including IFAD, invest a good deal of funding and effort in what are called ‘impact evaluations’ (RIMS level 3)  but which do not provide credible assessments of impacts.  At the same time, there is a great deal more that can be done to strengthen the basic output reporting and more qualitative assessments of how outputs are delivered and how the recipients respond to this. 

Regarding the first point on impact evaluation, the consensus in the room was moving towards what Chris Blattman, a Professor at Yale University advocates:  ‘Do R&D, not M&E’.  Essentially the idea is that impact assessment (level 3) should be focused on broader learning to determine what approaches generally work and under what conditions.  This requires a focused research program which develops carefully defined initiatives and employs rigorous, RCT-type evaluations of these initiatives conducted under different conditions in different regions (but not all projects!).  This  also requires an open attitude which does not focus on success or failure but rather on learning from the results whether they are what was expected or not (in fact, most of the great scientific advances in history were the unexpected, not predicted, results of experiments, but that is grist for another blog…) The objective is to generate something approaching generalizable conclusions regarding questions such as ‘ what is the best way to encourage increased use of fertilizer by smallholders to improve yields?’, or ‘what types of institutional conditions result in sustained empowerment of women in local, rural governance?’.   The result of this effort is an enhanced toolkit of ‘tested’ approaches and the conditions under which they should work which we can apply with some confidence of success.

The second point, regarding reporting, is that we need to focus more on monitoring to ensure that projects are doing the basics of transforming IFAD funding into goods and services for smallholder farmers, and, perhaps even more importantly, ensuring that these are of the right quality and delivered in a manner which meets the real needs of the target group.  Geoffrey Livingston, Sara Delaney and I wrote a paper about this for IFAD’s Smallholder Agriculture Conference in February of this year where we argued for much more attention to the challenges of spatial and temporal coordination of development activities, or more colloquially what we called ‘Right place, right time development’.  

So now for the modest proposal:  Let’s reorient our project M&E to  levels 1 and 2 where we can make a difference and use the information – specifically in monitoring the timely conversion of funding into goods and services for the target group through normal project reporting and due diligence.  Let’s then ensure that local universities or NGOs are hired to do annual ‘client satisfaction’ surveys regarding farmer’s views regarding the timeliness and quality of what they receive as well as how they are using the assistance provided.  As Geoffrey, Sara and I argued, simply addressing this part of the log frame or results chain would already likely lead to much better outcomes.

And what about impact studies?  Let’s ask those RCT-oriented members of the Board (and perhaps some interested foundations) to finance an internal IFAD fund for carrying out RCT and similar, high quality research on key issues for IFAD programs.  The IFAD fund could plan to provide finance and direct technical support for 2 to 4 RCT evaluations per year per Region which respond to priority issues for IFAD as a whole.  We can then share these results with partners, such as J-PAL, IFPRI, FAO, the World Bank and others in building up the collective rural development tool kit which program design could draw upon.  Of course we will continue to innovate and experiment beyond what is in the toolkit, but with more attention to designing these innovations in ways that facilitates evaluating if they work, and under what circumstances, we can strengthen the basis for the type of scaling up which is emphasized under IFAD 9.

And what will be the indicator for monitoring and evaluating this new M&E approach?  Simple - unlike Oakland Raider fans, we will ‘get over’ our frustrations with M&E and focus that energy on something more useful, like how we get effective agricultural finance working…. What do you think?

Participatory Monitoring and Evaluation Workshop – Days 5 and 6 – Banjul, The Gambia

This article is a contribution from Mr Lamin Fatajo, Monitoring and Evaluation Officer of the Rural Finance Project in The Gambia.

Finally after 4 days of interaction with eachother the group went out to the field and visited the villages of Sukuta and Brufut in western region. Prior to the field visit, roles of each actor were defined as well as the activities to be conducted. The objective of the field visit was to define the objective of Participatory Monitoring and Evaluation (PM&E) with the communities and identify the indicators they deem necessary to monitor for learning and value addition. The planning includes what will be done, how it will be done, the resources required and a timeline.
The PM&E Committees comprise of VISACA representatives, The Village Development Committee, a mentoring kafoo and 3 influential community members who were sensitized on PM&E in 2009.

After the fruitful field work, we all returned to Banjul to share the experiences gained from the field and also to determine the readiness of the trainees to conduct the step down training intended for the PM&E Committees at the village level.

The first step in the process was to explain the purpose of the visit, which includes an explanation of what PM&E is all about. Then the community defined their own objectives of the PM&E. This was centered on improving the performance of the VISACAs in both communities.

Objectives formulated with the population
- Increase advocacy for the VISACA System

- Improve linkages with other groups in the village to mobilize savings and membership

- Increase the number of savings and credit groups strengthen or formed.

The indicators to track the progress of achievement of these objectives were identified as change in membership, number of sensitization meetings conducted, change in group registration, change in the amount of deposits from groups and individual members of the groups, change in the number of groups formed/strengthen.

Some of the most valuable lessons learned during this exercise are:
- The exercise was highly participatory

- The purpose and objective of PME was well received and accepted by the communities

- The communities were able to identify their own priorities

- The PME committee can be used as an instrument for saving and membership mobilization

- There is hope that when project activities to be conducted are fully explained to the PME committees they can identify what they what from those activities and be able to monitor them

- Target setting is not fully embraced by the committee as show by only increase in the indicators without the magnitude of change expected.

- The trainees have handled the situation satisfactorily therefore, ready to conduct satisfactory PM&E training to the communities.

- Due to the dominance of the VISACA Management committees in the PME committees there is reason to believe that the purpose/objective of the PM&E was mainly for improvement of the VISACA system.

Innovations suggested by the committees during the exercise include:

- Targeting village based institutions such as schools and osusu groups (solidarity group financing)

- daily cash collection from market vendors

- operation of consumer cooperatives to fight inflation for VISACA members

On Saturday, the training workshop closed with the remarks of the acting project coordinator Mr. Sam Ali Ashcroft who thanked all the participants and WARF facilitators for a well attended workshop as well as recognizing the efforts of the WARF facilitator in ensuring that the workshop was a success. He also dilated on the useful role of the Multi-Disciplinary Facilitation Team (MDFT) members in the whole process particularly in ensuring the dentification of synergies that exist between different projects at the grass root level.

PM&E Worshop Banjul The Gambia - Day 4

While at dinner on Wednesday, we all had the pleasure of seeing ourselves on the Gambia Radio and Television Services (GRTS) evening news.

On Thursday morning, we resumed to our workshop with the presentation and analysis of the tools used in Participatory Monitoring and Evaluation, such as community scorecards, evaluation wheels, field force analysis, SWOT analysis. Each tool was explained, analysed and criticised by the participants.

The lively discussions brought us to the need for data transformation some cases and to discuss once again the difficulty of this exercise. Participants also agreed on the value added if the activity is undertaken properly and can truly improve the quality of results yielded by the projects. After a thorough analysis of the pros and cons of PM&E, the positive aspects of PM&E definitely outweigh the negative aspects. One of the most positive lessons to retain from these 4 days is that Monitoring and Evaluation is not the sole responsability of the Monitoring and Evaluation Officer,

but is the responsability of the entire project staff.

The afternoon was dedicated to setting up the field work to be done on Friday. A VISACA was to be visited and all stakeholders were to be involved in this preliminary process.

Unfortunately, I was not able to participate in the field work. I have asked my colleagues from the Rural Finance Project to write a little something regarding how the field work proceeded and the wrap up meeting held on Saturday morning. I hope to publish their contribution soon.

RFP-WARF Participatory Monitoring and Evaluation, Banjul, The Gambia - Day 3

Day 3 - Participatory Monitoring and Evaluation Workshop, Banjul, The Gambia

The 3rd day of the Participatory Monitoring and Evaluation (PM&E) had a particular agenda: it consisted in sort of a workshop within a workshop. The objective was to validate an M&E study undertaken for the project and was also the official opening of the workshop.

The meetings began with an official welcoming, where the acting Project Coordinator of the Rural Finance Project made an opening speech. His words were followed by an intervention by the Country Programme Manager for The Gambia and then by the welcoming of a representative of the Ministry of Agriculture who shared his views on the importance of M&E.

The rest of the day was dedicated to the presentation and critical review of the PM&E study conducted for the Rural Finance PRoject (RFP).

After a thorough presentation from one of the consultants who participated in the study to design the M&E system for RFP, the participants broke off into groups to discuss the overall valididty of the study by specifically looking at the roles identified, data collection methods, indicators identified and management of the system.

In the plenary, it was agreed that good preliminary work has been done, however, some more in debth work needs to be done on indicators (especially on 1st and 2nd level indicators) as the study is concentrated mainly on third level impact indicators.

The recommendation of the day is that the comments made on the report should be consolidated and submitted to the consultant and to WARF so that the report can be reviewed accordingly and finalised.

Tomorrow the training will continue and on Friday we are headed to the field to test what we have learned!

Participatory Monitoring and Evaluation Workshop - Day 2 - Banjul, The Gambia

The Rural Finance Project/West Africa Rural Foundation Participatory Monitoring and Evaluation (PM&E) workshop continues. The second day of the workshop has been interactive and lively even though it coincides with two national holidays, President Jammeh's Birthday - to whom will all wish a Happy Birthday - and African Liberation Day.

Participants were able to identify the key stakeholders and actors of the projects in The Gambia (project staff, beneficiaries and their organisations, government - national and local - service providers and the like) and reflected on their roles, responsibilities, rights, relationships and return in terms of project implementation and objectives. These actors were then placed into a power versus interest matrix. Much interesting debate sprung from this analysis especially regarding the roles and responsibilities and the power relationships that were identified.

The facilitators underlined how important these exercises are and insisted on the value added for projects in design phase, as these matrices clearly map out institutional arrangements and linkages and are a valid tool for later implementation.

More specifically, the matrices, if analysis is undertaken correctly, identify key partners in project execution (high on both the interest and power scales) and those in most need of capacity building to protect their interests (those high on the interest scale but low on the power scale).

The theme of the day was the importance of having clear and simple objectives, indicators and criteria and being able to communicate those to all of the actors involved in PM&E.


We then broke up into groups and engaged in role playing to identify objectives, indicators and criteria for projects. It was becoming quite obvious at this stage that agreeing on clear and simple objectives is not an easy task, especially when this is activity is undertaken with a participatory approach, and this was only a simulation!

Participatory Monitoring and Evaluation Workshop

Today, the FRAO/RFP Participatory M&E workshop was inaugurated in Banjul, The Gambia.

The Rural Finance Project in The Gambia (RFP) has organised with the support of the West Africa Rural Foundation FRAO/WARF a Participatory Monitoring and Evaluation (PM&E) training workshop in Banjul from 24 to 29 May 2010.

The objective of the workshop is to reinforce PM&E capacities of project teams and partners engaged in the implementation of IFAD funded projects in the Gambia in including VISACAS, service providers and government. Representatives of the IFAD Country Programme in The Gambia were present as well as representatives from the Ministry of Agriculture, the Microfinance Department of the Central Bank of The Gambia, Gambian Microfinance Associations (GAWFA, NACCUG) service providers and others.

The one week workshop has been set up in a participatory manner so that participants can share their views and learn from each other under the leadership of Mr Mohamed Kebbeh and Mr Adama Ndiaye of WARF who are facilitating the workshop.

Today, participants were able to brainstorm on Participatory Monitoring and Evaluation (PM&E) starting from its definition moving towards its importance in project implementation and the advantages and improvements it will bring to projects in terms of results and impact.

The lively group agreed that PM&E is a fundamental aspect for project implementation, as PM&E is not just a regular data collection or research activity, instead it increases ownership of the project for its inclusive nature, it gives a sense of accountability of project results to the communities in which interventions are taking place and it could improve project overall activities and outcome.

In summary PM&E is a shared learning process based on effective team work, necessary because of the large number of participants, negotiation of interests and possibilities and flexibility as the acceptance of PM&E implies a shift in power relationships because all participants are equal in PM&E.

Finally, PM&E allows for overall better results and gives continuity and sustainability to projects beyond their official life span thanks to beneficiaries designing the system based on criterion they have chosen rather than imposed indicators.

The key steps to PM&E are summarised as follows: (i) deciding to set up a PM&E; (ii) identifying the stakeholders; (iii) defining expectations and objectives; (iv) selecting criteria and indicators; (v) deciding on the tools and methods; (vi) collecting and analysing data; (vii) implementing change and manging and maintaining the system.

How to adapt his model to the context of Gambian projects and managing the system once its in place will be discussed during the course of this training.