This report provides a guide to evaluating the impact of a project or programme. Impact evaluation is about attributing impacts to interventions, and can play a key role in development effectiveness. No single analytical method is best for addressing all aspects of impact evaluations, but some methods have an advantage over others in addressing a particular question or objective. Different methods can complement each other to provide a more complete picture of impact. Impact evaluations provide the greatest value when there is an articulated need to obtain the information they generate.
In international development, impact evaluation is principally concerned with the final results of interventions (programmes, projects, policy measures, reforms) and their impact on the welfare of communities, households and individuals, including taxpayers and voters. It is important to understand that impact evaluation is about attributing impacts to interventions, rather than just assessing what happened. Further, identifying the impacts produced by an intervention requires an attempt to gauge what would have occurred in its absence.
A number of key issues need to be kept in mind when conceptualising and designing an impact evaluation:
- Identify the type and scope of the intervention. Two questions are useful here: the impact of what and the impact on what?
- Agree on the most important objectives of the intervention. Try to translate these objectives into measurable indicators while keeping track of important aspects that are difficult to measure.
- Consider the possibility of unintended effects and long-term sustainability of effects.
- Intervention design is based on a theory involving social, behavioural and institutional assumptions, and indicating why a particular policy intervention will work to address a given challenge. This theory needs to be articulated.
- Isolate and measure accurately the particular contribution of an intervention. Other interventions inside or outside the core area will often interact to strengthen or reduce the effects of the intervention.
- Use information from different approaches to assess different facets of complex outcomes or impacts. This will offer greater validity than using only one method.
- Build on existing knowledge relevant to the impact of interventions. Review and synthesis methods can help gather evidence to deepen the validity of an impact evaluation.
Impact evaluations can help to ensure that scarce resources are allocated where they can have the most impact. In managing impact evaluations, the following should be considered:
- Remember the importance of a clearly defined purpose and design, adequate resources, support from influential stakeholders and data availability.
- Start collecting data early. It is important to involve impact evaluation experts in intervention design as early as possible.
- Early planning is important to help manage the evaluation, its reception and its use. It is important to pay attention to country and regional ownership of the impact evaluation and to build capacity to understand and use it.
