Government budgets and their supporting systems – often referred to as Public Financial Management (PFM) systems – are one of the key tools that governments use to turn policy statements and intentions into the delivery of goods and services. In fact, much of what governments do depends on raising, borrowing and spending public resources. Interest in PFM systems and their performance is not a new phenomenon, but has become a lot more prominent in the past two decades for two main reasons: (1) the increased focus on institutions and governance as key determinants of countries’ development prospects, and (2) the growing use of ‘programme-aid modalities’ – like general or sector budget support – which channel foreign assistance directly through recipient countries’ budget systems.
Donor interest in supporting governance improvements and in ensuring that aid resources were effectively utilised to support poverty reduction – alongside the need to minimise fiduciary risk in budget support operations – led to the measurement of PFM systems’ quality and performance. While the UN and other actors had been looking at PFM-related issues for some time already, the first systematic attempt to measure the performance of PFM systems as a whole was promoted by the World Bank and the IMF as part of the HIPC debt reduction initiative in the late 1990s. The objective was to ensure that the resource freed up by debt cancellation could be effectively directed towards reducing poverty. WB/IMF teams applied the HIPC Public Expenditure Tracking Assessment and Action Plan (HIPC-AAP) tool in 20-25 HIPC countries over two rounds in 2001 and 2004 (see here). It looked at 16 indicators across the budget cycle and aimed to make sure that the resources freed up by debt cancellation could be effectively directed towards reducing poverty.
Building on the HIPC-AAP tool, a larger consortium of donors set up the Public Expenditure and Financial Accountability (PEFA) initiative a few years later. This initiative designed a more comprehensive assessment framework that has been widely utilised since its introduction in 2005 (see Reading 2 below). The PEFA framework has become the ‘gold standard’ when it comes to measuring the performance of PFM systems. It covers a wide range of topics and scores countries according to the degree to which they comply with recognised international good practices in each area. Assessments are usually carried out by teams of consultants, ideally (but not always) commissioned by country authorities, and are used to inform governments’ PFM reform strategies, the dialogue they have with donors about these and to guide donor decisions on choosing aid modalities.
A few other existing assessment tools look at the PFM system as a whole and are worth mentioning:
- The IMF’s Fiscal Transparency Code and related assessment tool –published in 1998, 2007 and 2015 – provides a comprehensive review of fiscal transparency practices and of how PFM systems guarantee fiscal discipline, promote good fiscal management and protect from fiscal risk (see Reading 2 below).
- The International Budget Partnership’s Open Budget Survey is an independent assessment of budget transparency practices that is carried out by civil society groups in 100 countries every two years (see Reading 3 below).
- The OECD’s Budget Practices and Procedures Database is based on a questionnaire filled in by government officials on the key characteristics of the budget process in their country. Although initially designed for OECD countries, it was applied in various regions, including Africa (see Reading 5).
- The World Bank’s Country Policy and Institutional Assessment provides yearly comparative scores for all low-income countries that are eligible for IDA lending. It includes an indicator on the quality of budget systems.
More specific ‘drill down’ tools have been developed for debt management, external audit, procurement, etc. A more complete list and comparison of different assessment tools – both general and specific – is available in Reading 1 below. It is important to highlight that the existence of country-specific documents can also be useful for measuring and monitoring the performance of PFM systems. These can include PFM reform strategies (see examples from Uganda and Albania) and reports published by internal and external audit institutions.
Different assessment tools have different characteristics and serve different purposes. As the readings below show, many of them have been used for both research and policy purposes. There are some questions relating to substance and process that are worth asking when looking at these different tools:
- On substance: What issues/topics does the assessment cover? How comprehensive is it? How detailed is the information it provides? What measurement criteria does it use? How factual/objective are they? Are the scores (if any) backed up by sufficient evidence?
- On process: How many countries are covered in the assessment? How often and how regularly are assessments carried out? How much do they cost? Who is involved in the assessment process? Are all the details and results publicly accessible?
Key readings
Reading 1: OECD/DAC. (2011). Stocktaking Study of PFM Diagnostic Instruments. Working Party on Aid Effectiveness/Task Force on PFM. Paris: OECD.
This study identifies, describes and compares internationally used analytical/ diagnostic tools in the Public Financial Management area, covering their objectives, coverage, frequency and cost, etc. It includes both general tools and tools which focus on specific PFM elements or institutions. Annex C in Volume I and Volume II provides useful comparative tables. The study also identifies existing knowledge gaps and provides a number of ideas for better alignment and harmonisation of assessment work of PFM country systems and greater government leadership.
Reading 2: PEFA. (2011). Public Financial Management. Performance Measurement Framework (revised version). Washington, DC: PEFA Secretariat.
This document contains the detailed methodology developed by the Public Expenditure and Financial Accountability (PEFA) initiative to assess the quality of PFM systems across countries. It includes 28 multi-dimensional indicators that span the various stages of the budget cycle, some cross-cutting themes and budget credibility issues, and three indicators on donor practices. The framework is currently being updated and revised, and a new methodology is already being piloted: http://pefa.org/en/content/pefa-2015-support.
Reading 3: IMF. (2014). Update on the fiscal transparency initiative. IMF Policy Paper. Washington, DC: International Monetary Fund.
This paper provides an update on the IMF’s efforts to revamp and re-launch its Fiscal Transparency Code, alongside a new assessment tool to be used to monitor countries’ compliance. This tool is divided into four pillars: (a) fiscal reporting; (b) fiscal forecasting and budgeting; (c) fiscal risk analysis and management; and (d) resource revenue management. Each pillar includes a number of areas being assessed, distinguishing among basic, good and advanced practices.
Reading 4: IBP. (2015). Open Budget Survey 2015. Open Budgets. Transform Lives. Washington, DC: International Budget Partnership.
This report provides a snapshot of the state of budget transparency across 102 countries. The results are based on a survey questionnaire that is filled in by independent researchers and looks at the public availability and comprehensiveness of eight key budget documents, at citizen participation in budget processes and at the strength of legislatures and audit institutions.
Reading 5: CABRI. (2009). Budget Practices and Procedures in Africa. Pretoria: Collaborative Africa Budget Reform Initiative.
This report highlights data from 26 African countries who participated in the OECD Budget Practices and Procedures questionnaire. It compares budget systems and practices across the African continent, looking issues such as budget calendars, fiscal transparency, off-budget spending and aid management.
Reading 6: Hedger, E. and de Renzio, P. (2010). What do Public Financial Management assessments tell us about PFM reform? ODI Background Note. London: Overseas Development Institute.
This note reviews different sources of data on the performance of PFM systems, assessing their pros and cons and analysing the resulting findings. It further considers the state of knowledge on factors that influence PFM reforms, including: country characteristics, reform processes and donor approaches.
Reading 7: Andrews, M. (2009). Isomorphism and the Limits to African Public Financial Management Reform. HKS Faculty Research Working Paper Series RWP09-012. Cambridge, MA: Harvard Kennedy School.
This paper uses PEFA data to investigate patterns in performance across PFM process areas by drawing on a dataset of disaggregated PEFA scores for 31 African countries, based on institutional theories of “isomorphism”.
Reading 8: P. de Renzio (2011). Buying Better Governance: The Political Economy of Budget Reforms in Aid-Dependent Countries. GEG Working Paper 2011/65. Oxford: Global Economic Governance Programme.
This paper combines analysis based on a new dataset on the quality of PFM systems across 16 developing countries (drawn from both PEFA and HIPC/AAP data) with in-depth case studies of Mozambique and Burkina Faso, to try and explain why in some countries PFM reforms have been more successful than in others.
Reading 9: Andrews, M.; Cangiano, M.; Cole, N.; de Renzio, P.; Krause, P. and Seligmann, R. (2014). This Is PFM. HKS Faculty Research Working Paper Series RWP14-034. Cambridge, MA: Harvard Kennedy School.
This short paper provides a basic overview of PFM systems, processes and actors. It argues that existing assessments tend to evaluate the degree to which PFM processes comply with forms considered ‘good practice’, at the expense of a focus on the following factors such practice should produce: (i) prudent fiscal decisions, (ii) credible budgets, (iii) reliable and efficient resource flows and transactions, and (iv) institutionalised accountability.
Questions to guide reading
- Looking at the various assessment tools that exist for measuring the performance of PFM systems, how would you define and choose some of the desirable characteristics of a “good” assessment?
- Most performance assessments are carried out by external experts or consultants, with some inputs by recipient governments and limited or no involvement by other stakeholders. How could assessments be organised so that they are more inclusive and participatory, serve a capacity building function and promote dialogue around reforms, rather than just respond to donors’ needs for performance information?
- Which of the various assessment tools is best suited for the following purposes: (a) promoting good fiscal management; (b) improving service delivery; (c) enhancing accountability; and (d) assessing fiduciary risk? Which specific indicators would you choose from the PEFA framework to assess performance against the different purposes?
- Compare the key findings from Readings 7 and 8. How do they differ? How are they similar? Which one do you find more useful?
- Do you agree with the argument that current PFM performance assessments are too focused on ‘form’ and not enough on the key ‘functions’ of PFM systems (reading 9)? What would you include in a list of key PFM goals and functions?
