The predominantly observational research assessing the impact of support to institutional reforms consistently finds mixed results. While there are some success stories, recent studies find that many institutional reforms do not seem to make government function better, often have quite poor results, and rarely lead to sustainable change and impact on poverty reduction (Andrews, 2013, p. xi). The dominant challenge in assessing impact remains one of measurement.
The disappointing results have led to questions on how best to support popular tenets of donors’ institutional reform agendas. Democratic decentralisation, citizen power for the marginalised, and social accountability may be intrinsically desirable, and there is some evidence of success particularly with community participation in development processes (McGee & Gaventa, 2010; Gaventa & Barrett, 2010). However, success depends largely on context-specific factors (Booth, 2012, p. 66) and the aid effectiveness agenda has proven difficult to operationalise: institutional reforms before and since the Paris Declaration do not look much different (Andrews, 2013, p. 211-212).
Thinking and working politically
- Booth (2014), drawing on successful aid-funded institutional reform in the Philippines, shows how PSIR efforts need to work within the political context and give up some traditional ways of working – including traditional project tendering modalities, notions of country ownership, and stakeholder inclusion. He notes that PSIR efforts need to be willing to give up strict control of the process, trusting local reform leaders to adopt their own strategies.
- Srivastava & Larizza (2013) show, with an example from Sierra Leone, how successful institutional reform involves identifying the political functions of prevailing institutions, to identify potential entry points.
Good governance agenda
- Andrews (2013, p. 15) provides quantitative and qualitative analyses in his book on institutional reform. In one example he finds that of 145 countries that undertook donor-supported institutional reforms from 1998 to 2008, 50% declined in government effectiveness as measured by the World Governance Indicators. For example, Senegal dropped by a quarter of a point, despite 27 projects with institutional reform content costing over USD 1 billion. A smaller set of 19 countries saw improved indicators.
- Scott’s (2011, p. 12) literature review of public sector governance reforms finds that evidence for assumed outcomes of decentralisation reforms is ‘largely missing’: qualitative research shows that reforms ‘have not had a consistent positive impact on service delivery, participation, poverty reduction or social cohesion’.
- However, other studies are more positive. Evidence from India suggests building more inclusive democracies may have a positive impact on development outcomes (Chattopadhyay & Duflo, 2004); Faguet (2004) finds that popular participation led to pro-poor outcomes in Bolivia; Scott (2011) reports evidence of local governance mechanisms opening up political space for women’s participation; and Enikolopov and Zhuravskaya (2006) find that when national political parties are strong, outcomes of fiscal decentralisation improve significantly.
- DFID’s 2013 Public Sector Management Portfolio Review found that policy management reforms, civil service reforms and human resources management reforms were the strongest performers for outputs in internal annual review scoring (2013b, p. 7). The review suggested that all programmes were too ambitious, aiming for improved performance throughout the public sector; they would work better if they were focused and had more realistic time frames. An incremental approach was recommended, combined with a long-term commitment to reform. When programmes did not work, political drivers and reform capacity were the main constraints.
- A multi-donor governance reforms evaluation examined the impacts of decentralisation reforms, centre-of-government reforms, accountability reforms, social justice reforms and the ‘whole of civil service reforms’ (SIDA et al., 2013). It found that the results were largely disappointing, with implementation, capacity and opposition to reforms being the main sources of derailment. The study concluded that reforms should emphasise sustainable changes, working with realistic theories of change; that political economy analysis should be used to understand potential impacts of reforms; and that success requires a whole of system approach, good donor coordination and a long-term perspective, which is difficult. Most reforms need better M & E systems that are embedded at the outset and focus not just on outputs, but also on outcomes.
- A recent review of the World Bank’s public sector management portfolio found that reforms worked better where the domestic political incentives were aligned to reform objectives (Blum et al., 2014). In general projects performed better in a) democratic rather than autocratic regimes; b) in aid-dependent countries; and c) in contexts with programmatic political parties. The review also finds that, given the importance of political factors, subjective risk assessments for reform were more reliable than standardised risk flags.
- A cross-country econometric analysis by de Renzio et al. (2011) finds that donor-funded public financial management (PFM) initiatives are positively but weakly associated with quality of PFM systems. The authors conclude that economic factors are most important in explaining differences in PFM system quality; aid-related factors have less explanatory power.
- An independent evaluation of AusAID’s law and justice institutional assistance finds many examples of effective approaches, but also frustration regarding the pace and extent of achievements (Cox et al., 2012).
- This consistent message is in spite of the challenges in evaluating the results of support to institutional reform. These challenges include: a lack of baseline information against which to measure success (which could be remedied by rigorous collection of sex-disaggregated data in PSIR programmes); difficulty in measuring outcomes (not easily captured by macro-level indicators of effectiveness and accountability) and impact (hard to establish clear and unambiguous evidence); and the lack of widely shared measures of reform activity or government effectiveness (Scott, 2011, p. 8; Andrews, 2013).
- Andrews, M. (2013). The limits of institutional reform in development. New York: Cambridge University Press. See document online
- Blum, J., Manning, N., & Srivastava, V. (2014). What predicts how World Bank public sector management projects perform? A review of the World Bank’s public sector management portfolio. Washington, DC: The World Bank. See document online
- Booth, D. (2012). Development as a collective action problem: Addressing the real challenges of African governance (Synthesis report of the Africa Power and Politics Programme). London: Overseas Development Institute. See document online
- Booth, D. (2014). Aiding institutional reform in developing countries: lessons from the Philippines on what works, what doesn’t and why (Working politically in practice case study 1). London: Overseas Development Institute. See document online
- Chattopadhyay, R. & Duflo, E. (2004). Women as Policy Makers: Evidence from a Randomized Policy Experiment in India. Econometrica, 72(5), 1409-1443. See document online
- Cox, M., Duituturaga, E. & Scheye, E. (2012). Building on local strengths. Evaluation of Australian law and justice assistance. Canberra : Ausaid. See document online
- de Renzio, P., Andrews, M., & Mills, Z. (2011). Does donor support to public financial management reforms in developing countries work? An analytical study of quantitative cross-country evidence (Working Paper 329). London: Overseas Development Institute. See document online
- DFID. (2013). How-to note: Capacity building. London: Department for International Development. See document online
- DFID. (2013b). Public Sector Management: A DFID Portfolio Review: 2008-2012. London: DFID.
- Enikolopov, R., & Zhuravskaya, E. (2006). Decentralization and Political Institutions (Working Paper No 65). Centre for Economic and Financial Research at New Economic School. See document online
- Faguet, J. P. (2004). Does decentralization increase government responsiveness to local needs? Evidence from Bolivia. Journal of Public Economics, 88(3–4), p. 867–93. See document online
- Gaventa, J., & Barrett, G. (2010). So what difference does it make? Mapping the outcome of citizen engagement (IDS Working Paper 347). Brighton: Institute for Development Studies. See document online
- McGee, R. & Gaventa, J. (2010). Review of impact and effectiveness of transparency and accountability initiatives: Synthesis report. Brighton: Institute of Development Studies. See document online
- Scott, Z. (2011). Evaluation of public sector governance reforms 2001-2011. Literature Review. Oxford: Oxford Policy Management. See document online
- SIDA, DFID, & Irish Aid. (2013). Summary report of the public sector governance reform evaluation. See document online
- Srivastava, V. & Larizza, M. (2013). Working with the grain for reforming the public service: A live example from Sierra Leone. International Review of Administrative Sciences. 79(3): 458-485. See document online