What is the validity and reliability of aggregate indexes of the quality of governance such as the Worldwide Governance Indicators (WGIs)? This study uses factor, confirmatory factor and path analysis to test both measurement and causal models of the six WGIs. Rather than distinguishing among aspects of the quality of governance, this paper finds that they appear to be measuring the same broad concept. The WGI authors should publicly release the sub-indicators to allow further investigation.
Governance perceptions indicators have become increasingly popular, but how accurate are they? In the case of the WGIs, some argue that there is no evidence for ‘construct validity’ presented for them – the constructs themselves are poorly defined and arguably meaningless.
This paper examines empirically the dimensionality of the WGI indexes to verify their construct validity. The study tests whether the six governance indicators measure a broad underlying concept of ‘effective governance’, or whether they are separate, causally related concepts. Findings are that:
- The indicators utterly fail to distinguish between the causal, measurement and mixed models. Variables that uphold all three models equally well are probably themselves not measuring what each claims to measure.
- Each WGI variable is represented as a ‘manifest’ (that is, measured) indicator. In reality none of the six indicators are conceptually distinct variables. Each WGI indicator is a constructed index comprised of numerous other manifest variables.
- There is no support in the data for several given ‘hypotheses’ about how indexes are conceptually related. Instead, a single factor explains nearly all of the variation in the six WGI indexes.
The results invalidate some practices in the research literature and by aid agency staff. Researchers generally treat the six WGI indexes as measuring distinct concepts. This study, in contrast, supports the choice of several researchers who have averaged together the six WGI indexes in their analyses into a single broader index. Therefore:
- Development practitioners should be cautious in their interpretation of the six WGI scores when diagnosing a country’s relative strengths and weaknesses in governance. As acknowledged by the WGI authors, the indicators, individually and collectively, are too imprecise for policy analysis and evaluation.
- Public release of the individual sub-indicators that comprise each one of the six WGI indicators would allow investigators to examine whether the WGI indicators are as distinct as their nomenclature suggests. Even though many of the sub-indicators of each WGI indicator are based on perception, they are all based on surveys that may be indicative of different aspects of governance.
- Though each sub-indicator may lack the reliability of the aggregate indicators, they may still provide important and perhaps more precise information about different aspects of governance. Assessing the six WGI indicators is ultimately not a measurement task, but six separate causal hypotheses that many investigators are actively investigating. That the relation between institutions and outcomes is often endogenous does not make them the same thing.
