The selection and design of climate change mitigation and adaptation interventions should be based on evidence of what works (and what does not), under what circumstances and at what cost. Currently, evidence on the impact of such interventions appears limited, and there is a strong case for the wider application of rigorous impact evaluation. New studies should evaluate positive and negative impacts of climate change interventions on both environmental and welfare outcomes. Programme planners and evaluators should work together to accommodate rigorous impact evaluation from the start. While appropriate outcome indicators will differ between interventions, future evidence syntheses will be improved by work to develop a consensus on a set of common outcome indicators.
Impact evaluation (IE) measures the net change in outcomes for a particular group of people that can be attributed to a specific programme. It involves using different methods to create robust comparison groups (counterfactual analysis), or using ‘natural experiments’. In IE, the estimated impact of the intervention is calculated as the difference in mean outcomes between a treatment group (those receiving the intervention) and a control group (those who do not). The single difference estimator compares mean outcomes at end-line and is valid where treatment and control groups have the same outcome values at baseline. The difference-in-difference (or double difference) estimator uses baseline and end-line data to calculate the change in outcomes over time across the two groups.
Climate change policy appears to be behind other policy areas in IE. Apart from a few quasi-experimental evaluations in the related field of conservation, the application of IE to climate change interventions has been limited. Further, only two of these studies include estimates of both environmental and welfare outcomes. However, in order to establish why an intervention has been successful, theory-based IE is increasingly being used to examine the causal chain, and to test the underlying assumptions of a programme’s theory.
Evaluating the impact of environmental programmes is challenging:
- Such programmes often lack baseline data and a theory of change that includes causal relationships.
- There is often a time lag between the intervention and a measurable impact.
- The main focus of interventions has been in areas which are not easily evaluated (such as governance and institutional processes)
- There is limited experience in IE method and design.
- IE faces numerous problems and limitations, including sampling issues, spill-over effects and imprecise survey data.
However, existing and proposed evaluations indicate that both experimental and quasi-experimental IE approaches can be applied to climate change adaptation and mitigation interventions, including in sectors such as agriculture, water, social protection and disaster risk reduction.
Despite the challenging nature of climate change intervention IEs, they should become a key component in promoting the efficient use of resources. Evidence for the effectiveness of current spending is essential to give credibility and support for the massive finance required for future mitigation and adaptation. IEs are urgently required to establish which interventions are most cost-effective, and to identify both positive and negative impacts on environmental and developmental outcomes.
