Are field experiments useful in studying the impact of development assistance on democracy and governance (DG)? This article reviews recent and ongoing DG field experiments. It considers the prospects for and obstacles to the development of a useful body of experimental evidence on the political economy of development. There are significant challenges related to the difficulty of generalising from small samples and micro-level projects. However, although the field experiments have tended towards village-level interventions, they suggest that impact evaluations can potentially address higher-level interventions and theories.
Some donor agencies hope that randomised controlled trials (RCTs) can provide robust evidence of impact to inform programme decisions and justify resource allocations for DG projects. Academics recognise that field experiments offer powerful tools for testing arguments about the drivers of political change in developing countries. The field experiments generate high internal validity because they address the problem of selection bias with random assignment of target and control units. However, the challenges of conducting field experiments lead researchers to focus on certain types of interventions, and limit the types of evidence collected.
The article reviews field experiments that assess a) DG outcomes of social or economic development assistance; b) social and economic outcomes of DG promotion programmes; and c) DG outcomes of DG programmes. Most field studies evaluate interventions at the level of a village or neighbourhood (at a local school, clinic or police station). This narrow focus limits the lessons that one can accurately learn from RCTs. For example, unique features of a village can interfere with the ability to generalise from micro-level results. Mechanisms for decision-making that work well among acquaintances may not yield optimal results beyond such intimate settings. Lessons about citizen participation may not translate into best practices for building accountability at higher levels of government. Further challenges include the following:
- DG programmes tend to combine multiple activities targeted at diverse beneficiaries. This multifaceted approach makes it difficult to test hypotheses.
- Measuring outcomes is also more difficult with complex interventions. Evaluation may not offer the precise evidence about cause and effect needed for theory building and policymaking.
- Attempts to simplify programmes to facilitate evaluation can generate a body of evidence about atypical programmes. The same is true if researchers gravitate toward the study of simple programmes or unusual sites where there are few programmes.
- Programme objectives, activities and hypotheses are only well specified at implementation time. Therefore, measured impact may not reflect real impact. Collection of data on inputs, outputs and proximate outcomes reveal only imperfect clues about the actual nature of implementation.
In order to move towards more generalisable evidence, experiments need to be replicated in multiple contexts. Other issues to consider include the following:
- Recognising the current limitations to the evidence produced by field experiments can help scholars and practitioners to identify appropriate domains for generalising theoretical and programmatic findings.
- A careful mapping of the conditions under which all similar programmes are implemented versus those under experimental investigation would help both academics and practitioners understand the limitations of valid inference.
- Experimental investigations of local government programmes can offer better tests of theories of representative government than interventions at the community level.
- A focus on interventions involving higher levels of local government would offer greater opportunities for generalisation. Middle levels of government offer settings that are more analogous to national governments.
