How can the quality and effectiveness of developing countries’ national procurement systems be measured? This paper produced jointly by the World Bank and the OECD Development Assistance Committee (OECD-DAC) outlines the agreed methodologies for measuring 12 indicators related to national procurement systems. The indicators are designed to assist governments in developing their own procurement capacities and to enable donors to mitigate risks in the projects they fund. Four national procurement “pillars” are evaluated: legislative and regulatory framework; institutional structure and management capacity; procurement operations and market practices; and systems’ integrity and transparency.
The “Johannesburg Declaration” of December 2004, under World Bank and OECD-DAC auspices, expressed international commitment to the Baseline Indicator (BLI) tool, which evaluates aspects of individual countries’ procurement systems against international standards. Subsequently, the Joint Venture for Procurement developed methodologies for measuring Compliance / Performance Indicators (CPIs), which focus on the practical application of the BLIs.
BLIs are measured through reviews of regulatory frameworks and institutional and operational arrangements; CPIs are assessed via data obtained from contracts and interviews with stakeholders in the procurement system. Although the methodologies are intended to provide objective, easily-comparable assessments, some indicators are not amenable to statistical measurement and certain CPI data may be difficult or costly to obtain.
Assessors give scores out of three for countries’ performance in each BLI, which is also associated with a specific CPI. Each BLI is grouped into four main pillars, and sub-divided into smaller sub-indicators, whose score can be assessed according to specific criteria:
- Pillar I, “legislative and regulatory framework”, covers the extent of the legal framework over the procurement process. Assessors evaluate the scope of regulatory frameworks during tenders, the quality of operational procedures and the availability of model tender documentation and conditions of contract.
- Pillar II, “institutional framework and management capacity”, examines the practical functioning of the legal framework through the country’s public sector governance systems. It focuses on the procurement system’s degree of integration into the governance structure, the efficacy of any regulatory body and the existence of bodies to monitor procurement system performance.
- Pillar III, “procurement operations and market practices” looks at the efficiency of the market for procurement competition. It comprises indicators measuring procurement agencies’ efficiency, the market response to public procurement solicitations and the quality of contract administration practices after awarding of contracts.
- Pillar IV, “integrity and transparency of the public procurement system” measures anti-corruption powers within the system and the extent to which stakeholders can oversee the system. It includes indicators related to the countries’ control and audit systems, the efficiency of their appeals mechanism, the availability of information on procurement and the existence of anticorruption measures.
- CPIs cannot be assessed numerically as there are no agreed standards for these indicators. However, they are important elements of procurement monitoring which can be analysed in narrative reports that outline the extent to which a country’s performance meets its own regulations or what the quantitative perception of compliance is.
