• About us
  • GSDRC Publications
  • Research Helpdesk
  • E-Bulletin
  • Privacy policy

GSDRC

Governance, social development, conflict and humanitarian knowledge services

  • Governance
    • Democracy & elections
    • Public sector management
    • Security & justice
    • Service delivery
    • State-society relations
  • Social Development
    • Gender
    • Inequalities & exclusion
    • Social protection
    • Poverty & wellbeing
  • Humanitarian Issues
    • Humanitarian financing
    • Humanitarian response
    • Recovery & reconstruction
    • Refugees/IDPs
    • Risk & resilience
  • Conflict
    • Conflict analysis
    • Conflict prevention
    • Conflict response
    • Conflict sensitivity
    • Impacts of conflict
    • Peacebuilding
  • Development Pressures
    • Climate change
    • Food security
    • Fragility
    • Migration & diaspora
    • Population growth
    • Urbanisation
  • Approaches
    • Complexity & systems thinking
    • Institutions & social norms
    • PEA / Thinking & working politically
    • Results-based approaches
    • Theories of change
  • Aid Instruments
    • Budget support & SWAps
    • Capacity building
    • Civil society partnerships
    • Multilateral aid
    • Private sector partnerships
    • Technical assistance
  • M&E
    • Indicators
    • Learning
    • M&E approaches
Home»Document Library»When Will We Ever Learn? Improving Lives Through Impact Evaluation

When Will We Ever Learn? Improving Lives Through Impact Evaluation

Library
Center for Global Development
2006

Summary

Despite decades of investment in social development programmes, we still know relatively little about their net impact. So why are rigorous social development impact evaluations relatively rare? This paper from the Center for Global Development (CGD) aims to address this question and provide recommendations for more and better evidence for policymaking and programme planning. A new, collective approach is needed, in which developing country governments, bilateral and multilateral development agencies, foundations and NGOs work together to define an agenda of enduring questions and fund the design and implementation of rigorous impact evaluations in key sectors.

Although significant resources are devoted to monitoring of programme outputs or process evaluations, there is insufficient investment in impact evaluations. Reviews of evidence that are conducted in the course of policy or programme design routinely reveal that little is known about “what works” and decisions are made on the basis of anecdotal or other weak evidence. Of those studies that have been completed, many are methodologically flawed so that net impact cannot be estimated in a valid way.

Three basic incentive problems need to be overcome to generate more and better impact evaluations:

  • A portion of the knowledge generated through impact evaluation is a public good. Such broad benefits are amplified greatly when the same type of programme is evaluated in multiple contexts, and addresses enduring questions. However, the cost-benefit calculation for such an evaluation might not include these benefits, making it look like the impact evaluation is not worthwhile.
  • The rewards for institutions, and for individuals within them, come from “doing” not from ‘building evidence’ or ‘learning’. Thus, it is extremely difficult to protect the funding for good evaluation, or to delay the initiation of a project to design the evaluation and conduct a baseline study. The opportunity costs are seen as too great. Consequently, resources that might be devoted to rigorous evaluation are used instead for project implementation.
  • There are disincentives to finding out the truth. If policymakers and programme managers believe that future funding depends directly on achieving a high level of success, the temptation will be to concentrate on producing and disseminating ‘success stories’.

The solution to overcoming these incentive problems lies both in strengthening capacity within organisations and in creating a collective approach. Specifically:

  • Organisations should commit individually to strengthening monitoring and evaluation systems, dedicating resources to impact evaluations and building capacity in developing countries.
  • These efforts by individual organisations should be coupled with collaborative – collective action among actors to coordinate and fund rigorous impact evaluation.
  • A committee, standards-based network, secretariat, or other organisation is needed as a focal point to lead a collective action initiative. This ‘council’ should include any developing country governments, development agencies, NGOs, foundations, and other public and private entities willing to engage actively in improving the quality and increasing the quantity of impact evaluations.
  • The core functions of the council would be: to establish quality standards; administer a review process for evaluation designs; identify priority topics; and provide grants. Other functions would include: organising and disseminating information; building capacity to produce, interpret, and use knowledge; creating a directory of researchers; and undertaking communication activities.

Source

Center for Global Development, 2006, 'When Will We Ever Learn? Improving Lives Through Impact Evaluation', Evaluation Gap Working Group, Centre for Global Development, New York

Related Content

Lessons from stabilisation, statebuilding, and development programming in South Sudan
Helpdesk Report
2020
Doing research in fragile contexts
Literature Review
2019
Designing, Implementing and Evaluating Public Works Programmes
Helpdesk Report
2018
Indicators and Methods for Assessing Entrepreneurship Training Programmes
Helpdesk Report
2018
birminghamids hcri

gro.crdsg@seiriuqne Feedback Disclaimer

Outputs supported by FCDO are © Crown Copyright 2023; outputs supported by the Australian Government are © Australian Government 2023; and outputs supported by the European Commission are © European Union 2023
Connect with us: facebooktwitter

Outputs supported by DFID are © DFID Crown Copyright 2023; outputs supported by the Australian Government are © Australian Government 2023; and outputs supported by the European Commission are © European Union 2023