People are not generally open to change and ideas that contradict what they already believe. Under certain conditions, they actively avoid such information while at the same time seeking information that bolsters their original beliefs.

Research organizations and the people that work for them are not immune from this. Many who spent their careers in research or international development resist the idea that their efforts may be ineffective or even counterproductive. Cognitive dissonance theory predicts that, based on levels of commitment to current beliefs, evidence to the contrary will be rejected and even discussion (for learning) is discouraged.

In a world of rapid change (such as we have in our Livestock and Fish program) where value chains actors interact in complex and unpredictable ways, “traditional” monitoring approaches that are heavy and slow, infrequent and top-down are simply inadequate. Real-time monitoring and quick (learning) feedback loops, faster cycles of data collection and analysis, allow for quick assessment of positive and negative effects of interventions and help immediate course-correction to ensure that research is more responsive to the needs of smallholder farmers and their constituencies.

Exploring Theories of Change, the interfaces of capacity development with change processes, social psychology to understand research (uptake) and development practices, and studying ways to overcome them will have little benefit if the broader research and development field is not predisposed to carefully listen to evaluation findings.

Research organizations may unwittingly be complicit in undermining national and local capacity development, creating attitudes or expectations that actually weaken well-intended work for development, replacing local resourcefulness and self-reliance with attitudes that view the benefits of externally-directed programs as an entitlement that hinders national (public, private sector) ownership and new initiatives. Failure to recognize how on-the-ground implementation change processes and dynamics fosters such attitudes may help explain why many of the improvements attributed to externally-funded programs persistently lack sustainability. Why have many “modern” research (for development) programs, in general, left their partners/clients inert, dis-empowered, and uncommitted to act independently on the challenges they face?

Finding ways to create space for meaningful dialogue on this is a most significant challenge.

A “new” research for development paradigm has supplanted a long-functioning, traditional system of addressing farmer community needs. While those pre-existing systems may or may not have been efficient or equitable, they rarely left communities inert, unwilling to take action, and dependent on others. This may not be the result of a lack of capacity of communities to manage new technologies or systems, but rather an unwillingness to accept responsibility for them.

The amount of guidance and support farmer communities receive from researchers and development partners is often inversely related to the even short term sustainability of an initiative. This has profound implications for development practice: sustained development happens when external development agents intervene less and when national systems are built, and local participation and ownership is encouraged. Unfortunately, failure to empirically ground these concepts in the social psychological field of attitude change and behaviour have led to widely divergent interpretations of their meaning and more importantly, to their ineffective implementation in the field.

To emphasize this point, “community participation” was claimed to be a key, albeit diversely understood element within each of the development processes leading too often to deleterious outcomes. If extensive external intervention does undermine sustainability, there are issues with the nature of these interventions that a better understanding of social psychology could help address.

Let me give you some space to reflect about what I just said….

The last decade has seen the rise of behavioral economics and a growing interest in using it, together with behavioural psychology, to understand how behaviour influences change (or not) in day-to-day life and work of people, as well as the functions of value chains. Yet research innovation (for developmental change) has, in my opinion, not really caught up with these developments and understandings of what motivates individuals and organizations, and how to harness that motivation. There is a need to get more granular in our understanding of how to provide a stimulus for change in different parts of the value chain, for different types of staff, and for diverse kinds of technological innovations, from incremental to radical.

This week I am attending a workshop organized by our CGIAR Research Program on Livestock and Fish to refine frameworks and tools for a new Monitoring and Evaluation (M&E) system based on the program’s Theory of Change and Impact Pathways, aiming to ensure common understanding of theories, to discuss how they can be used for planning, critical reflection and accountability; and to develop change pathways for the pilot value chains in which it will be tested.

That was a mouthful of jargon, isn’t? Let me try to make some sense out of it for you.

Strengthening capacities of collaborators and partner institutions is crucial to CGIAR’s mission. In the Strategic Results Framework (SRF), and the CGIAR Capacity Development Framework, capacity strengthening is recognized as a key strategic pillar along with research to increase its impact of research and adoption of innovation.

Currently, CGIAR centres and the CGIAR Research Programs (CRP) do not have systematic ways of monitoring, tracking, and reporting capacity development – change – activities. In the absence of a systematic database for capacity strengthening activities, the assessment of CGIAR’s and any CRP’s performance with regard to its capacity development – change – objectives becomes difficult. Further, merely counting the number of people who attend (as an example) training courses may not fully capture the change process supported by CGIAR researchers at the three systems level (individual, organizational, and institutional).

To determine whether our research program is effectively contributing to developing national (research) capacity, the program must have a comprehensive system to assess whether a) CGIAR’s capacity development principles are properly integrated into value chain design/planning, b) if flagship programs and value chains do invest and implement ways that are consistent with the CGIAR capacity development approach, and c) whether concrete capacity development results are systematically captured and reported following a capacity measurement framework.

Learning about Theories of Change for the Monitoring and Evaluation of Research Uptake (IDS, 2014) is a clear imperative for us to innovate for change in the program. This argues that program staff and partners will determine the success or failure of getting research for change to the heart of development (processes and implementation). The literature of capacity development converges around a few tenets that represent challenges to Innovations in Monitoring and Evaluating Results (UNDP, 2013) and the measurement of change.

• First, effective capacity development entails significant elements of self-directed change driven and owned by (for example) national research institutes, smallholder farmers. As a result, aspects of the work cannot always be defined before the process is begun.
• Second, CGIAR usually supports capacity development to improve performance within a wider system, rather than as an end in itself. There is generally an underlying theory of change (stated or implicit) that presumes that capacity development activities will strengthen certain stakeholders and modify attitudes and practices, in turn changing the performance of a wider system. Consequently, appropriate data should be monitored to track each of the steps in this theory of change, capability gains, behavior changes, and performance improvements, with the emphasis placed more on the performance improvements, and the link of improved performance to higher-order results.
• Third, capacity development involves investments in capabilities with uncertain future applications. It is difficult to predict the impacts of a stronger human resources process, better communications, or a more widespread culture of learning. These areas represent difficulties for measurement because they may produce a, value chain, partner organization that is more nimble, adaptive, or resilient in the face of change and responsive to uncertain challenges or opportunities – challenges or opportunities that may or may not arise. These adaptive capabilities are vital to the performance of individuals, organizations, and institutions, but are difficult to monitor through pre-established indicators. Staying abreast of holistic capacity development programming requires thus additional monitoring methods to complement indicator tracking.
• Fourth, capacity development depends on the partner’s skills, incentives, commitment, resources, and opportunity costs around processes of reflection, prioritization, absorption of new skills or knowledge, and putting new priorities, skills and knowledge into practice.

The Livestock and Fish program anticipated that capacity development assistance would be provided somehow through intermediary (development) organizations. Such intermediaries can be held accountable for the quantity, quality, timeliness, and relevance of their assistance, but not for the behaviour changes of their partner/client organizations. Indicators for the performance of intermediary organizations in capacity development should focus on what they are responsible for doing within their manageable interests, as distinct from the actions of those they assist – though this can include some joint accountability for collective performance together with those they assist.

So, what frameworks can help us to identify different areas of potential changes in partner capabilities and performance over different time scales? For example, a new strategic outreach plan might affect a local organization’s efficiency quickly, but only increase influence over a longer period. It is clear that the new functions or roles any organization, network, or individual wants to take on depend critically on the space for it to do so in the broader system in which the organization operates. It is therefore critical to indicate selection to understand the wider context that affects changes in individual and institutional attitudes, practices, and behavior.

The first step is to articulate the purposes of the capacity development component of specific value chain program/projects and the Theory of Change. Here, brainstorming about potential measurements by identifying the specific capacity, behavior, and performance changes we expect to see (as well as areas where we are unsure what to expect, but want to monitor change if it occurs) needs to take place.

The second step is to select one or more core purposes of the monitoring of any Theory of Change:

• Tracking capacity development interventions: Monitoring the inputs provided by CGIAR centers, partner organizations, and others as well as the outputs produced from the inputs. Inputs and outputs are relatively easy to measure and track. Input indicators include the activities and resources used for capacity development;
• Tracking capacity development sub-purposes or outcomes: Monitoring changes in behavior by partner/client organizations and resulting changes in their performance over time. These changes generally take more time to achieve than inputs and outputs. Examples of outcome (or sub-purpose) topics to monitor include changes in the organization’s legitimacy in eyes of stakeholders or influence in eyes of decision-makers through perception surveys, a service delivery organization’s efficiency is improved (reaches more farmers for the same cost), percentage of organization’s proposals that are funded, level of technological research uptake;
• Accountability of (potential) capacity service providers: Monitoring the productivity, quality, and timeliness of capacity development service providers. Service provider accountability often is measured through a mix of input indicators and triangulation. Examples might include number of people mentored, number of organizations actively participating in joint learning network, and customer satisfaction surveys of recipients of capacity development services.

It is quite common to have multiple purposes. However, it is usually not possible to monitor all of these purposes through one indicator or approach. Having clarity on the purposes behind our monitoring allows us however to interrogate the purposes of capacity development interventions in our value chain, and help prioritize in which of those areas we will require monitoring approaches.

A learning approach is critical to enable adaptive management for capacity development. Monitoring and evaluation are both vital in supporting a learning approach, particularly where CGIAR and its partners can engage in joint review of jointly-defined indicators. Monitoring can track what has changed and link that back to a theory of change. Monitoring is most likely to support effective capacity development when centers, implementing partners, and client organizations collaborate on definitions of indicators and targets and joint reviews are conducted to support mutual learning and adaptation. An evaluation would be needed to gain a better understanding of how and why the theory of change worked or did not work. Evaluations can also consider unintended consequences, alternative explanations, and lessons learned in greater depth.

Follow me on twitter: @DianaBrandes