Page contents
- The case for participatory M&E
- Challenges in using participatory approaches
- Participatory tools: case studies
Monitoring and evaluation (M&E) has typically been led by outside experts, measuring performance against pre-set indicators and using procedures and tools designed without the participation of key stakeholders such as the programmes’ intended beneficiaries. Evaluations in particular, because they are very often conducted by external consultants, can be seen as a form of control.
For a number of years, there has been widespread recognition that M&E should take a more inclusive, participatory approach. Participation in this sense means the involvement of stakeholders in deciding how the project or programme should be measured, in identifying and analysing change, and in acting on results. Nevertheless, there are few empirical studies of the effectiveness, quality and objectivity of participatory approaches. Questions such as how to operationalise participation and which methods work in which contexts are the subject of ongoing debate.
The case for participatory M&E
Proponents of participatory monitoring and evaluation (PM&E) argue that it is more cost-effective, accurate and sustainable than conventional approaches. Participation in decision-making processes can also motivate people to want to see those decisions implemented effectively. Another motivation for PM&E is to strengthen organisational and institutional learning.
Since the 1990s there has been growing innovation in the use of quantitative approaches to PM&E. A range of methods and tools have emerged including counting, mapping, valuing and scoring. While these approaches have proved effective in some contexts, their success in empowering communities ultimately remains dependent on official attitudes and acceptance.
Chambers, R., 2007, ‘Who Counts? The Quiet Revolution of Participation and Numbers’, IDS Working Paper No. 296, Institute of Development Studies, Brighton
How do participatory approaches generate numbers? A range of participatory methods have been developed, many visual and tangible, by which local people themselves produce numbers. This paper provides an overview of the field, building a case for more widespread use of participatory numbers in development practice and research. It argues that in many contexts, processes generating participatory numbers are better than traditional questionnaires. They provide greater validity, better insights, and increased knowledge gains for local people.
Access full text: available online
Challenges in using participatory approaches
Whilst the ideological case for participation is widely acknowledged, PM&E is not without its critics. Crucially, labelling M&E as ‘participatory’ does not necessarily guarantee that all stakeholder groups have participated, and there are often issues around who participates and who is excluded from these processes. Subsequently, the representativeness of the findings and recommendations of participatory evaluations have been criticised.
Gregory, A., 2000, ‘Problematizing Participation: A Critical Review of Approaches to Participation in Evaluation Theory’, Evaluation, vol. 6, no. 2, pp. 179-199
It is widely accepted that evaluation is a social process which implies the need for a participatory approach. But what is understood by ‘participation’? This review argues that the blanket use of the term has masked the heterogeneity evident in its realisation in practice, and highlights a lack of transparency in participatory evaluation methods.
Access full text: available online
Operationalising PM&E can also be problematic. There is a need to be sensitive to the local socio-economic and political situation, and consider under what conditions PM&E approaches can be used without increasing the vulnerabilities of already marginalised groups.
Estrella, M., 2000, ‘Learning from Change: Issues and Experiences in Participatory Monitoring and Evaluation’, International Development Research Centre, Ottawa
Since the 1980s, concepts of participatory monitoring and evaluation (PM&E) have entered the policymaking domain of larger donor agencies and development organisations. This introductory chapter draws on twelve case studies to describe how different stakeholders have applied PM&E approaches across a range of purposes and contexts. It outlines some of the key concepts and differences between participatory and conventional approaches to M&E and highlights emerging issues.
Access full text: available online
Participatory tools: case studies
There is no single definition or methodology of PM&E, and it encompasses a wide range of tools and approaches. Common among these approaches are values such as shared learning, democratic processes, joint decision making, co-ownership, mutual respect and empowerment. Below are some examples of participatory tools and how they have been used in practice.
Participatory Rural Appraisal
Participatory Rural Appraisal (PRA) encompasses a broad range of methods to enable local people to analyse their own realities as the basis for planning, monitoring and evaluating development activities. PRA uses group exercises to facilitate information sharing, analysis, and action among stakeholders.
Chambers, R., 2007, ‘From PRA to PLA and Pluralism: Practice and Theory’, IDS Working Paper, no. 286, Institute of Development Studies, Brighton
How have the theory and practice of participatory methodologies in development activities changed since the mid 1970s? What variants and applications of these methodologies have emerged? This paper traces the spread of Participatory Rural Appraisal (PRA) and Participatory Learning and Action (PLA), and identifies strengths and weaknesses in the ways they have been implemented. Whilst inflexible applications of PRA and PLA may produce disappointing results, when executed with spontaneity and creativity, these approaches can be a source of community empowerment.
Access full text: available online
Beneficiary Feedback
Beneficiary feedback systems seek to collect key stakeholders’ views about the quality and impact of a development agency’s work. This approach has grown in popularity and has been supported by a range of donors. While beneficiary feedback systems are likely to improve sustainability and can empower beneficiaries, they may present only a partial impression of beneficiaries’ views, and there has been a lack of rigorous evaluation of their impact. While there is growing interest in the role that new information and communication technologies can play in scaling up PM&E techniques, these advances do not provide a ‘magic bullet’ to improving participation.
Jacobs, A., 2010, ‘Creating the Missing Feedback Loop’, IDS Bulletin, Vol. 41, No. 6, pp.56-64
How can development agencies implement feedback systems so as to hear systematically from the intended beneficiaries of their work? How can such systems improve aid accountability, and thus effectiveness? This article focuses on agricultural development, arguing that in most agricultural development projects, quantified summaries of smallholder farmers’ views can be collected. Such data can provide real-time performance indicators that incentivise staff to focus on beneficiaries’ priorities. The feedback process can be empowering in itself, and acting on the feedback can significantly improve project impact and sustainability. While measuring performance against plans encourages supply-led and agency-centred development, using feedback systems can encourage demand-led and people-centred development.
Access full text: available online
GSDRC, 2010, ‘Participatory M&E and Beneficiary Feedback’, Helpdesk Research Report, Governance and Social Development Resource Centre, Birmingham
PM&E and beneficiary feedback approaches can improve effectiveness and sustainability and reduce the costs of monitoring, but risk generating unrepresentative results and increasing tensions between stakeholders. PM&E should be understood as social and political processes: PM&E and beneficiary feedback mechanisms should be tailored to the local context and integrated into local political structures and processes.
Access full text: available online
Key Informant Interviews
Key informant interviews are a rapid assessment methodology that can be used as an intermediate indicator of outcomes as an alternative or supplement to full impact assessments.
Price, N., and Pokharel, D., 2005, ‘Using Key Informant Monitoring in Safe Motherhood Programming in Nepal’, Development in Practice, vol. 15, no. 2, pp. 151-164
The Nepal Safer Motherhood Project (NSMP) works to improve maternal health and contribute to programme development at district and national level. This article discusses the project’s use of Key Informant Monitoring (KIM). KIM is an adapted version of the peer ethnographic research method. Data is collected by community-based Key Informant Researchers (KIRs) and used for monitoring and planning. KIRs have proved useful sources of information and acted as change agents by spreading safer motherhood messages.
Access full text: via document delivery
Most Significant Change Technique
The Most Significant Change (MSC) technique involves the collection of change stories from the field level, and the systematic selection of the most significant of these. These selected stories are then discussed and critically reflected on to help determine the impact of the development programme or activity.
Davies, R., 2005, ‘The ‘Most Significant Change’ (MSC) Technique: A Guide to its Use’, MandE, London
This paper outlines an innovative qualitative monitoring technique known as the ‘most significant change’ (MSC) approach. The MSC technique is a participatory method of collecting and analysing stories from the field which focuses on monitoring intermediate outcomes and impact. It provides a simple means of making sense of a large amount of complex information and is best suited to large-scale, open-ended projects that would be difficult to monitor using traditional methods.
Access full text: available online
There is evidence that the most significant change technique can enhance organisational learning and performance.
Wrigley, R., 2006, ‘Learning from Capacity Building Practice: Adapting the ‘Most Significant Change’ (MSC) Approach to Evaluate Capacity Building’, INTRAC Praxis Paper no. 12, International NGO Training and Research Centre, Oxford
There is growing recognition of the need to take a multi-stakeholder approach to evaluation, which promotes local ownership and builds capacity for reflection, learning, improved performance and self-determination. This paper reflects on the use of the ‘Most Significant Change’ (MSC) methodology to evaluate the capacity building services of CABUNGO, a local capacity building support provider in Malawi.
Access full text: available online
Outcome Mapping
Outcome Mapping is an alternative to theory-based approaches to evaluation that rely on a cause–effect framework. It recognises that multiple, non-linear events lead to change. It focuses on people and changes of behaviour and how far development interventions have built the capacity of the local community. Outcome mapping assumes only that a contribution has been made, and never attempts attribution.
Earl, S., Carden, F., and Smutylo., T., 2001, ‘Outcome Mapping: Building Learning and Reflection into Development Programs’, International Development Research Centre, Ottawa
This paper argues that impacts are often the product of events for which no single agency can claim full credit. Outcome mapping moves away from impact assessments to focus on changes in the behaviour of the people it works with directly.
Access full text: available online
The following paper reviews the use of outcome mapping, examining its key advantages, the contexts in which it is most useful and how donors can facilitate its use. It finds that outcome mapping works best in situations where the implementing agency is working in partnership with a range of other stakeholders, is seeking to build capacity, or when tackling complex problems.
Jones, H. and Hearn, S., 2009, ‘Outcome Mapping: A Realistic Alternative for Planning, Monitoring and Evaluation’, Overseas Development Institute, London
What is Outcome Mapping (OM) and why is it valuable? When does it work best? How can donors facilitate its use? This note draws on case studies to review OM – a flexible, actor- and learning-centred approach to planning, monitoring, and evaluating social change initiatives. It finds that adopting OM for appropriate projects could help development agencies to increase their effectiveness and meet commitments to managing for results. OM is well-suited to areas involving complex change processes, capacity building work, and knowledge and decision-making processes. Shifting to OM’s learning-oriented mode requires donors to adopt more realistic expectations and to dispense with the idea of ‘controlling’ change processes. Crucially, OM must be underpinned by real trust between the donor, project implementers and partners.
Access full text: available online
The Outcome Mapping Learning Community is an informal community that allows practitioners to share knowledge and experiences. A number of case studies are available on its website: www.outcomemapping.ca.
