The literature widely notes the importance of setting up monitoring and evaluation systems to facilitate feedback loops and continuous learning to enable reprioritisation and revision of the sequence of activities (World Bank, 2011b). The prioritisation and sequencing of activities often happens during the implementation phase, rather than in the prior planning phase, so the literature suggests continuous analysis and flexible programming (CIC, 2011). Typically in FCAS, there is monitoring and evaluation of frameworks (see Box 15), the donor’s country engagement and sector-level engagement, and individual projects and programmes (DFID, 2010d).
Box 15: Example of a monitoring tool
Iraq – Joint Monitoring Matrix (JMM) (2006)
The government of Iraq and the UN launched the International Compact with Iraq in 2006. A Joint Monitoring Matrix (JMM) detailed and sequenced goals with timelines. After one year, an annual review of the compact lead to a significant reduction to the number of priorities to reflect a revision of what was seen as achievable in the timeframe. The content of the compact was also adjusted. One of the lessons to emerge from this experience was that ‘compacts should be tangible, implementable, focused, and based on government capacity to deliver’.
Source: Bennett, C. (2012).
Theories of Change
Theories of Change (ToCs) are important to programming in FCAS because of the political dynamics and risks involved in bringing about change. ToCs provide a testable hypothesis, can help make theories explicit, and articulate assumptions about how change can occur and the impacts that certain actions will have (Woodrow, 2013). ToCs are widely viewed as essential elements of the design phase, and the monitoring and evaluation phase.
ToCs are important for monitoring and evaluation processes as they can provide feedback on whether programmes are on track to achieve desired changes, and whether the context is evolving as anticipated. ToCs are also useful for monitoring assumptions to help determine if the right factors and dynamics were considered in the initial design, if unforeseen changes have occurred in the environment, or if there are gaps in the strategy to bring about change (Corlazzoli & White, 2013a).
Strengths and weaknesses
Despite the importance of monitoring and evaluation, CIC (2011, p.4) identifies it as the ‘weakest link’ in strategic planning in FCAS. Key challenges include: political will; capacity; data collection; attribution; methodologies (e.g. weak theoretical foundations and evidence base); the high risk of violence; complex and unpredictable contexts and interventions; politicisation and multiple actors (CIC, 2011; OECD, 2012b, p.27). Bakrania (2014, p.36) identifies the following challenges specific to using ToCs: ToCs can encourage oversimplification; gathering evidence to test ToCs is difficult; and programme designers often have implicit or explicit ToCs in mind that are not communicated.
As there is no one sequence that a country should follow, there is also no one definition of what ‘successful sequencing’ or ‘successful prioritisation’ would be, or how it could be measured. While donors have become increasingly effective at producing cogent analyses, donors are frequently weak at applying these analyses to their programmes (CIC, 2011; Yanguas & Hulme, 2014). Issues to consider in monitoring and evaluation include: how diagnostic tools have been used to inform the design and adaptation of activities; how frequently the analyses have been updated; how much buy in/participation analyses garnered across all actors involved (including the different donors, and the different sections within each donor).
OECD (2012a, p.29-31) suggests the following core principles for evaluation in FCAS:
- Context as the starting point / conflict analysis: Conflict analysis is central to evaluation of donor engagement in FCAS as it can be used to assess whether activities have been adequately sensitive to the conflict setting, determine the scope of the evaluation (what will be evaluated), and identify evaluation questions.
- Conflict sensitivity: Conflict sensitivity is the ability of an organisation to: (1) understand the context it is operating in; (2) understand the interaction between the intervention and that context; and (3) act on that understanding to avoid negative impacts and maximise positive impacts on the conflict (CDA, 2009 in OECD, 2012a, p. 29). The paper recommends that evaluators should be aware of the impact the evaluation could have on the conflict, and on the safety of the people involved (e.g. interviewees, interpreters, staff in country). They also emphasise that conflict sensitivity does not automatically deliver an effective peace programme or policy, and that the next step is evaluating conflict sensitivity (and its effectiveness).
- See GSDRC Topic Guide on Safety, Security and Justice, p.34-36.
- For more information about conflict sensitivity see the GSDRC Topic Guide on the subject and Saferworld’s Toolkit: Conflict-sensitive approaches to development, humanitarian assistance and peace building: tools for peace and conflict impact assessment.
- Bakrania, S. (2014). Safety, security and justice: topic guide. Birmingham, UK: GSDRC, University of Birmingham.
See document online - Bennett, C. (2012). Aid Effectiveness in Fragile States: Lessons from the First Generation of Transition Compacts (Policy Paper). New York: International Peace Institute.
See document online - Center on International Cooperation. (2011). Strategic Planning in Fragile and Conflict Contexts. New York: CIC, New York University.
See document online - Corlazzoli, V., & White, J. (2013a). Practical approaches to theories of change in conflict, security and justice programmes: Part 2: Using theories of change in monitoring and evaluation. London: DFID / Search for Common Ground
See document online - DFID. (2010d). Working Effectively in Conflict-affected and Fragile Situations: Briefing Paper I: Monitoring and Evaluation. London: DFID.
See document online - OECD. (2012a). Evaluating Peacebuilding Activities in Settings of Conflict and Fragility Improving Learning for Results. Paris: OECD-DAC.
See document online - OECD. (2012b). Coherent planning and prioritisation. In International Support to Post-Conflict Transition: Rethinking Policy, Changing Practice, pp. 31-43. Paris: OECD.
See document online - Woodrow, P. (2013). Practical approaches to theories of change in conflict, security and justice programmes: Part 1: What they are, different types, how to develop and use them. London: DFID/CDA. See document online
- World Bank. (2011b). World Development Report 2011. Washington, DC: World Bank.
See document online - Yanguas, P., & Hulme, D. (2014). Can aid bureaucracies think politically? The administrative challenges of political economy analysis (PEA) in DFID and the World Bank (ESID Working Paper No. 33). Effective States and Inclusive Development Research Centre, University of Manchester.
See document online