Are criticisms of the Worldwide Governance Indicators (WGI) tenable? This paper from the World Bank summarises and responds to recent critiques of the WGI. It argues that such criticisms are misguided as they misinterpret the indicators, lack empirical support or apply a conceptual framework which is ill-suited to the practical aim of the WGI.
The WGI have reported on perceptions of governance in over 200 countries since 1996. They aggregate data from 31 sources, provided by 25 organisations, and organise them into six governance dimensions. Their usefulness derives from the breadth of their country coverage, the diversity of data sources and their adoption of a transparent margin of error. As a result, the WGI have now become one of the most widely-used set of governance indicators.
Criticisms of the WGI relate to the following five areas:
- The WGI are ill-suited to comparisons over time and between countries. Such comparisons could be distorted by the diversity of data sources and the fact that changes may merely reflect corrections of past errors. Comparisons may also be impeded by the lack of precision in the WGI and the fact that indicators are scaled to produce the same global average in each period.
- The WGI are analytically biased. Since commercial organisations provide some of the data, the results are slanted towards the priorities of business elites. They also contain ‘halo effects’ – they are overly influenced by countries’ recent economic performance and overall level of development.
- Correlated errors in the various data sources distort the WGI. Since some sources are interdependent, the benefits of aggregation are diminished. Replicated errors in results may also be given excessive weight in the WGI.
- The WGI lack ‘construct validity’, in that many of the definitions employed are inadequate. Categories used to group governance data lack clear definition and are insufficiently distinguishable from each other.
- The WGI are insufficiently transparent. Some of the data informing the indicators are unavailable to other researchers, thus impeding evaluation, replication and peer review.
However, these criticisms can be rebutted on empirical or conceptual grounds:
- The re-scaling of WGI averages to produce a ‘zero mean’ for each period does not impede comparison as there is little evidence for general changes in world governance levels over time. The aggregation methodology successfully merges different data sources into comparable units. Error corrections are not systematic and imprecise results simply reflect the inevitable limitations in governance data.
- Several non-commercial data sources complement the business data. In fact, the perceptions of businesspeople differ little from other groups. The evidence for ‘halo effects’ is weak and may merely reflect the causal relation between economic success and good governance.
- It is unlikely that interdependent data sources will produce exactly correlated errors. Even if sources are based on similar data or reports, their aggregated correlation may merely be evidence of accurate results.
- Some definitional ambiguity is inevitable given the nature of governance assessment and should not hinder data analysis. In fact, the categories selected demonstrate good empirical coherence and distinguishability.
- Most of the data employed in the WGI is in the public domain and reproduced in the reports themselves. Although the availability of the Country Policy and Institutional Assessment ratings to the public is limited, broader criticisms of the WGI’s transparency are exaggerated.
