How is evaluation managed and resourced in development agencies? What are the major trends and challenges in development evaluation? This report reviews evaluation in 23 bilateral aid donors and seven multilateral development banks. It finds that significant progress has been made towards creating credible, independent evaluation processes. Challenges include the need to improve collaboration with other donors and country partners and to support the use of findings and take-up of recommendations. The use of evaluations needs to be systematised, for instance, by integrating the consultation of relevant reports into the planning process for new programmes or country strategies. Lessons from evaluations need to be better formulated and targeted to specific audiences in accessible and useable ways.
Evaluation in development agencies is moving away from a focus on project outputs to assess broader impacts of development assistance. It is adapting to new aid modalities and cross-cutting issues, and is increasingly involving country partners. Emerging trends also include the diversification of actors involved in evaluation, and more co-ordination among donor evaluation departments.
Most development agencies and multilateral development institutions have a single central evaluation unit overseeing programme-level evaluations. These units tend to be responsible for strategic evaluation at country, sector or thematic level and for wider questions of development effectiveness, institutional performance, strategic learning and accountability. Single project-level evaluations have largely been incorporated into performance or results-based management schemes in programme units. However, links between decentralised evaluations at the project or country level and the main central evaluation unit at headquarters appear weak or ill-defined in many agencies.
To enhance independence, most evaluation units rely on consultants to conduct evaluations. However, there is concern with the quality of consultants. In fact, around half of the agencies reported concerns over inadequate resourcing for high quality and rigorous impact evaluation. There was widespread acknowledgement of insufficient human resources and of weak technical skills, particularly in new or complex areas of aid, such as budget support, governance or conflict. Other findings include the following:
- There is substantial use of quality assurance systems, and management response systems to evaluations have been established in about half of the agencies studied. However, few mechanisms are in place to monitor, assess or transparently report actions taken as a result of evaluation.
- While donors do consult other donors’ evaluation plans, joint working represents only 36 percent of evaluations for bilaterals (15 percent for multilaterals), mainly undertaken by small groups of like minded donors.
- Interactions with diverse stakeholders have deepened, but there is little genuine joint working with partner countries; they are mainly involved in data gathering tasks.
- Just over half of reporting members of the OECD DAC Network on Development Evaluation work to support evaluation capacity development in partner countries, though most lack a clear strategy for these efforts.
Key issues for future consideration include how to use evaluation results to improve development effectiveness, feasible approaches to involving partner countries, and the effective management of evaluation resources. Improvements in evaluation quality and credibility do not necessarily result in increased use of the findings: there is growing concern around evaluation communication and use. A system for management to formally respond to evaluations has emerged as a potentially useful way to ensure that evaluation findings are incorporated into programme management and policy development. It is also important to:
- Produce targeted lessons and recommendations for specific audiences, make better use of new technologies and talk candidly about success and failure
- Enhance the technical skills and specialised knowledge of personnel in agencies and partner countries and of consultants
- Involve development partners more centrally in the evaluation process: both in programme design and in using the results and ensuring that follow up action is implemented.
