What are good indicators and how can they be selected? While results-based approaches are gaining popularity with donors and partner countries, there remains a question of whether they deliver higher returns than traditional aid modalities. This paper argues that the quality of indicators may determine the success of results-based approaches; the link between disbursement of funding and results means that indicators need to be adequately selected and create the right incentives to be achieved. Drawing on findings from a cross-country analysis of indicators from pilot projects in the education sector, it makes a number of recommendations to donors and partners in order to strengthen and ensure the quality of indicators and their measurement.
The paper conceptualises a typology of indicators and devises a criteria for assessing the quality of indicators. The key criteria are: focus on results; control; financial incentives; measurability and verifiability; and unintended consequences. These criteria are then applied to assess the quality of indicators used in five education pilot-projects in Ethiopia, Rwanda, Sri Lanka and Tanzania. Two of these projects are implemented by multilaterals, three by bilaterals (DFID), and one by a local NGO. The education sector was chosen because it is already far advanced in implementing results-based approaches and the selected pilot projects offer a cross-country perspective.
The report makes 5 tentative recommendations. Results-based approaches should:
- Rely on outcome-indicators for making disbursements. Donor and partner countries should try to be as innovative as possible, but where pre-financing or little certainty on the results-chain exists, a gradual approach with stronger reliance on process indicators could be applied.
- Emphasise results orientation over attribution concerns. Incentivised actors should have plausible control over achieving the measured activities and results. It should be possible to demonstrate that the results-based approach is one of main causes of observed changes in indicators – using outcome indicators should guarantee this.
- Prioritise leverage effects over value for money. Funders should put more emphasis on potential leverage effects and risks for partners. If VFM is the main criterion, a significantly reduced incentive effect may result.
- Reward incremental improvements rather than set targets. Linking disbursements to the achievement of certain thresholds is not advisable – targets set too low or too high do not create strong performance incentives. Similarly, funders should not set annual ceilings.
- Draw on national systems and administrative data for measuring results. Parallel administrative data systems are costly and undermine country ownership, complementing in-country systems with independent verification will ensure impartiality.
- Use direct indicators and a robust verification process to avoid unintended consequences. It is important that results-based approaches are only implemented if good measures exist to minimise unintended effects. To avoid distortions, partners need to be able to significantly influence results with genuine efforts. Independent verification of results is also crucial to avoid data manipulation.
