What does impact mean in relation to accountability programmes and projects? This paper argues that current approaches to impact assessment in this field are inadequate: methodological wars are overshadowing key issues of power relations and politics. A learning approach to impact assessment is needed that gives power and politics a central place in monitoring and evaluation systems. Instead of looking at the extent to which the desired impact was achieved, it is important to look at what happened as a result of the initiative, how it happened and why. It is also important to test and revise assumptions about theories of change continually and to ensure the engagement of marginalised people in assessment processes.
Current approaches to studying the impact of accountability fail to consider the politics of accountability. Impact assessment tends to be viewed as a way of demonstrating the proper use of funding and delivery of results, rather than as a way of learning about how change happens and how to get better at working with change. This mismatch is often mistaken for a methodological clash between the proponents of deductive experimental evaluations and inductive qualitative approaches. Contradictions have also arisen between the essence of governance and accountability work and the results-based mindset of development aid:
- Governance programmes are messy and non-linear, involving reversals as well as gains in a context of entrenched interests. Multiple stakeholders are involved in producing change, working in alliance or coalition (via contestation and political manoeuvring). Thus, change occurs over long periods of time in complex configurations of power which are highly context-specific.
- However, efforts to demonstrate the results of governance programmes tend to be based on linear, logical models of change. They seek to attribute impact to single causes, and they measure symptoms rather than underlying problems. They also press for quick returns on programme investments and ‘value for money’ in the assessment exercise itself.
The relationship between transparency and accountability is not a given, and projects require an explicit theory of change (currently often absent). Jonathan Fox’s ongoing work on ‘accountability politics’ suggests a basis for a theory of change, while also highlighting that the power relations and contestation involved in accountability processes make outcomes unpredictable. Fox suggests that successful TAIs are characterised by four processes:
- Civil society is activated, generating an interface with state actors or entry into a governance space.
- Horizontal accountability actors (from ‘inside the [government] agency’, from ‘outside the agency and inside the state’ and from ‘outside the state’) are activated, and form alliances with citizens and CSOs.
- These allies disempower the vested interests that oppose transparency and accountability. This involves moves and countermoves by many parties for and against different kinds of accountability and transparency, in shifting alliances.
- Partial success stimulates further civil society action.
Complex political processes like citizen-led accountability initiatives require complex, power-aware ways of assessing their impact and understanding how it has occurred. A key question is: what happened to the power relations that needed to change? Further:
- Programme design needs to be informed by nuanced power and political economy analysis, painstaking baseline analysis of the power relations that prevailed at the outset, and tools for regularly revising and updating that initial analysis
- Participatory monitoring and evaluation approaches need to be revisited and adapted to enable the relatively powerless to define what constitutes favourable change in the balance of power in their context
- Indicators or milestones need to capture reconfigurations among the relevant actors in society and between them and the relevant actors in ‘the state’
- It is important to understand that steps forward are likely to be followed by a hardening of resistance that will set back progress towards impact.
