Graphic outline of the meeting sessions that will lead to an operational Monitoring, Evaluation and Learning framework for the program, 27 November 2013 (photo credit: ILRI/Jo Cadilhon).

The CGIAR research program on Livestock and Fish held a Monitoring, evaluation and learning (MEL) planning meeting last week in Limuru, close to Nairobi, Kenya. The MEL team of the International Livestock Research Institute (ILRI) had invited colleagues involved in the Livestock and Fish program from ILRI and from other CGIAR partner centres. The main objective of the meeting was to collect feedback from colleagues on the MEL framework and achieve a consensus on determining evaluation indicators.

The MEL framework document spells out how the program will monitor and evaluate its progress to reaching the intermediate development outcomes (IDOs) it has fixed itself, and how the program will learn from this process to produce change. I was invited to facilitate the meeting and I wish to share a few lessons I took away from it.

What is monitoring, evaluation and learning?

The first activity after the introduction of participants was to answer the following question:

In 3 words – To my mind, the Livestock and Fish Monitoring, evaluation and learning framework is…

The responses from the 18 participants were a testimony to the different viewpoints people can have about MEL. Here is how I would classify these responses.

For some reason, an evaluation is often seen as threatening. Some of the responses captured this uneasy feeling: ‘Accountability’, ‘A requirement’, ‘Inspecting’, ‘Evaluation’.

On a more positive side, others saw monitoring, evaluation and learning as a helpful process: ‘How change happens’, ‘How we’re doing’, ‘How we learn’, ‘Learning, questioning’, ‘Our road map’, ‘Targets, people’, ‘Guiding measurement’, ‘Tracking progress’, ‘Important for future’.

Finally, a third set of opinions placed MEL in the bigger picture of the Research Program and the CGIAR’s work: ‘Overarching’, ‘Very important’, ‘Slowly being developed, ‘Getting clarified today’ ‘A 3-D jigsaw’, ‘Part of CGIAR strategy’, ‘It’s about IDOs’, ‘Impact’, ‘Measuring impact’.

One could see from the start of the meeting the diversity of viewpoints held by participants on what monitoring, evaluation and learning is all about.

Rome wasn’t built in a day

At the end of the first day, I asked participants to tell me the main lesson they had learned from Day 1.

Many responses highlighted the difficulty of capturing the complexity of the multi-stakeholder system that is Livestock and Fish into one monitoring and evaluation framework. To deal with this complexity, we came to an agreement that the group would have to make difficult choices and that it was essential to document the pros and cons of these choices as we went along.

But I wish to highlight the fact that 8 out of 18 participants had realized how long and complex the process of developing a MEL framework for a CGIAR research program would be: ‘Still a lot of work’, ‘Still need to agree on overarching issues’, ‘Worried of losing learning’, ‘There is no perfect indicator’, ‘MEL team still has a lot of work to do, ‘Cannot do it in a day’, ‘Cannot model reality perfectly’, ‘All need to contribute to this MEL framework’.

I think this meeting was extremely useful because it allowed the various stakeholders of the program to understand the implications on their work of choosing this or that method of evaluation and this or that evaluation indicator. I think many also realized the difficulty of distilling a very complex system into a limited set of easily tracked indicators. Evaluation indicators should be carefully thought out and agreed by stakeholders and partners involved in the program to be evaluated. In that sense, it was a pity that out of the program partner centres, only Malcolm Dickson from WorldFish attended the meeting.

Achieving consensus

In hindsight, this one-and-a-half-day meeting has already been a mini evaluation all on its own: despite the various viewpoints of the different Livestock and Fish stakeholders involved, consensus was achieved on pinning down a method and tools that would allow the research program to evaluate its impact according to the intermediate development outcomes it has set itself. In the results of the final evaluation form, one can read some frustration: issues dear to some have not been captured or are still missing from the framework. Nonetheless, all participants now have a clearer view of the program MEL framework and how they will fit in it. To quote one of the answers to ‘What have we achieved during this meeting?’: ‘Greater team spirit and ownership awards results’.

ILRI collaborates in other CGIAR Research Programs which also need to elaborate their monitoring, evaluation and learning framework, and to identify indicators to measure progress towards intermediate development outcomes. The three lessons highlighted above are useful ones for all these complex and multi-stakeholder systems.

Jo Cadilhon, Senior Agro-Economist, ILRI Policy Trade and Value Chains Program