Attempts to go beyond purely quantitative measures – such as numbers of participants – require that more robust qualitative measures of these concepts are introduced into assessments. Measures of impact also need to take into account unintended or secondary impacts which occur as a result of greater empowerment and accountability.
McGee, R., and Gaventa, J., 2011, ‘Shifting Power? Assessing the Impact of Transparency and Accountability’, Working Paper No. 383, Institute of Development Studies, Brighton
What does impact mean in relation to accountability programmes and projects? This paper argues that current approaches to impact assessment in this field are inadequate: methodological wars are overshadowing key issues of power relations and politics. A learning approach to impact assessment is needed that gives power and politics a central place in monitoring and evaluation systems. Instead of looking at the extent to which the desired impact was achieved, it is important to look at what happened as a result of the initiative, how it happened and why. It is also important to test and revise assumptions about theories of change continually and to ensure the engagement of marginalised people in assessment processes.
Access full text: available online
Solava, I., and Alkire, S., 2007, ‘Agency and Empowerment: A Proposal for Internationally Comparable Indicators’, OPHI Working Paper Series, Oxford Poverty and Human Development Initiative, Oxford
The benefits of empowerment for deprived people have often been put forward without the backing of a large and well-established body of empirical research. This article proposes indicators that could be added to individual or household surveys to generate internationally comparable data. Using such data, researchers could improve our understanding of connections between variables (such as empowerment and income, governance, health and nutrition outcomes) in different contexts and of their durability over time. The proposed indicators include: control over personal decisions; domain-specific autonomy; household decision-making; and the ability to change aspects of one’s life at the individual and communal levels.
Access full text: available online
Jupp, D. and Ibn Ali, S., with Barahona, C., 2010, ‘Measuring Empowerment? Ask Them – Quantifying Qualitative Outcomes from People’s Own Analysis’, Sida Studies in Evaluation 2010:1, Sida, Stockholm
How can empowerment be measured? This paper presents the experience of a social movement in Bangladesh, which found a way to measure empowerment by letting the members themselves explain what benefits they acquired from involvement and by developing a means to measure change over time. These measures have also been subjected to numerical analysis to provide convincing quantitative data which satisfies the demands of results-based management. The study shows how participatory assessments can empower and transform relationships, while at the same time generating reliable and valid statistics for what were thought to be only qualitative dimensions.
Access full text: available online
Graham, C., 2010, ‘The Challenges of Incorporating Empowerment into the HDI: Some Lessons from Happiness Economics and Quality of Life Research’, Human Development Research Paper 2010/13, UNDP, New York
Access full text: available online
Kabeer, N., 2001, ‘Resources, Agency, Achievements: Reflections on the Measurement of Women’s Empowerment’, pp 17-59 in Discussing Women’s Empowerment – Theory and Practice’, Swedish International Development Cooperation Agency, Stockholm, Sweden
In efforts to measure empowerment, do indicators mean what they are supposed to mean? Are the values they reflect appropriate? This paper examines the measurement of three dimensions of empowerment: resources (the conditions under which choices are made); agency (the process by which choices are made); and achievements (the outcomes of choices). It highlights problems of meaning and values – particularly the need for indicators to be triangulated – and suggests that methodologically pluralist approaches to measuring empowerment are required.
Access full text: available online
McDevitt, A., 2010, ‘Helpdesk Research Report: Measuring Women’s Economic Empowerment’, Governance and Social Development Resource Centre, University of Birmingham
It appears that the majority of efforts to measure women’s economic empowerment programmes focus on quantitative outcomes – such as increased access to credit or increased revenue – even where the stated objectives include broader empowerment goals. Whilst some evaluations include variables to show that women have not been disempowered, few succeed in showing that specific aspects of women’s power have actually increased.
Access full text: available online
USAID, IFPRI, and OPHI, 2012, Women’s Empowerment in Agriculture Index, USAID, IFPRI, and OPHI
This new index measures the roles and extent of women’s engagement in agriculture in: 1) decisions about agricultural production, 2) access to and decision-making power over productive resources, 3) control over use of income, 4) leadership in the community, and 5) time use. The WEAI was developed by the US government, the International Food Policy Research Institute and the Oxford Poverty and Human Development Initiative.
Access full text: available online
The following paper presents the World Bank’s framework for conceptualizing and measuring empowerment. This sees empowerment as resulting from improvements in an individual’s personal agency and opportunity structure. Agency refers to an individual’s capacity to make purposive choices, and is measured by asset endowments, which can be psychological, informational, organizational, material, social, financial, or human. Opportunity structure is the institutional context in which choice is made, and this is measured by the presence and operation of formal and informal institutions, including the laws, regulatory frameworks, and norms governing behaviour.
Alsop, R. and Heinsohn, N., 2005, ‘Measuring Empowerment in Practice: Structuring Analysis and Framing Indicators’, World Bank Policy Research Working Paper 3510, World Bank, Washington D.C.
How can we determine whether and how projects and policies aimed at empowering stakeholders reach their intended goals? Empowerment is recognised by the World Bank as one of the three pillars of poverty reduction, and is found in the documentation of hundreds of its projects. This paper presents an analytic framework that can be used to measure and monitor empowerment processes and outcomes. It argues that the framework is useful both within single countries and for cross-country comparison of degrees of empowerment.
Access full text: available online
A growing number of studies have sought to use Randomised Control Trials to measure empowerment and accountability processes and programmes. The following report outlines some of the key challenges associated with using RCTs for these programmes and notes that scholars are divided about the ability of these methods to generate reliable results.
Walton, O., 2011, ‘Helpdesk Research Report: RCTs for Empowerment and Accountability Programmes’, Governance and Social Development Resource Centre, University of Birmingham
This report examines the extent to which RCTs have been used successfully to measure empowerment and accountability processes and programmes. Field experiments present immense opportunities, but the report cautions that they are more suited to measuring short-term results with short causal chains and less suitable for complex interventions. The studies have also demonstrated divergent results, possibly due to different programme designs. The literature highlights that issues of scale, context, complexity, timeframe, coordination and bias in the selection of programmes also determine the degree of success reported. It argues that researchers using RCTs should make more effort to understand contextual issues, consider how experiments can be scaled up to measure higher-order processes, and focus more on learning. The report suggests strategies such as using qualitative methods, replicating studies in different contexts and using randomised methods with field activities to overcome the limitations in the literature.
Access full text: available online
Existing approaches to measuring Voice and Accountability are also useful.
Holland, J., and Thirkell, A., 2009, ‘Measuring Change and Results in Voice and Accountability Work’, DFID Working Paper 34, DFID, London
How can the often intangible results of Voice and Accountability (V&A) interventions be measured? This paper adapts DFID’s ‘Capability, Accountability and Responsiveness’ (CAR) governance framework for use with V&A work. It maps existing indicators onto this adapted framework, developing a menu of V&A indicators and data collection instruments. Measures need to take account of the costs as well as benefits of poor people’s voices being heard, and should reveal the obstacles to poor people’s engagement. V&A indicator data can effectively combine observable and measurable changes in behaviour with perception scoring of the quality of those changes.
Access full text: available online
Mcloughlin, C., 2009, ‘Helpdesk Research Report: Voice and Accountability Indexes’, Governance and Social Development Resource Centre, University of Birmingham
This report identifies tools, indicators and indexes that have been developed to measure the impact of development interventions on voice and accountability. In terms of tools, theory-based approaches are prevalent, and there is some consensus about the value of participatory evaluation and mixed-methods designs combining surveys and interviews. It was not possible, however, to identify any cases of the use of rigorous impact evaluation methodologies to assess changes in voice or accountability as the result of donor interventions. In relation to indicators, there does not appear to be a single set of combined indicators for assessing voice and accountability – indeed this may not be possible or desirable. The World Bank‘s Voice and Accountability Index is widely cited. Indexes commonly draw on indicators of participation, budget transparency, access to information, freedom of the press, the status of women, and citizens‘ views of the quality of democracy.
Access full text: available online
