This paper describes the pilot of a mixed-method research instrument designed to measure the empowerment impact of social policy. Research was conducted in three working-class communities in Jamaica (two urban and one rural) to determine young people‘s sense of empowerment in their relations with the police. The project used two instruments to generate data: (i) a Community Score Card (CSC) completed by focus groups and (ii) rapid assessment peer interviews of individual young people in the communities (ethnographic research).
Findings from the CSC tool was that while indicators were useful for measuring the accountability gap, they can be misleading—and even dangerous—if they reduce power relations to a depoliticized relationship between service provider and user.
The analytical lens for the study was based on two fundamental assumptions about human behaviour. First, that everyday interactions are framed by adaptive preferences, in which people‘s awareness of what is desirable or possible is shaped (and limited) by habitual interactions. The implication for empowerment is that when people have very low expectations of what they are entitled to, a survey of their attitudes, perceptions, and expectations will not necessarily provide guidance for improving a given service.
The second assumption is that in order to create an enabling environment for empowerment (i.e., to enable people‘s capacity to envision and demand change) through policy interventions, it is essential to broaden the focus from the immediate locus of a given service or interaction (in this case, youth-police relations) and instead address it within the broader context of political, economic, and social empowerment.
Key finding:
- Qualitative data generated by the peer ethnographic research was critical for achieving a deeper, more nuanced understanding of the scorecards findings, broadening the context of youth-police relations by looking at social institutions in terms of identity, authority, and power relations. Findings from the peer ethnographic research often appeared to contradict CSC scores (especially the more positive ones). This could be due to either the added value of probing the habitual responses of the focus groups or to the unreliability of scores given by a small sample. The only way to assess the relative significance of these two explanations would be to elicit perception scores through individual surveys of larger populations, either alongside CSCs administered to focus groups (if the local process remains important) or instead of group-based CSCs.
- Although the outcome of the exercise constitutes a strong case for the added value of using mixed-method tools for policy analysis, it is too early to say whether its application in Jamaica has had a tangible impact on policy. It does appear, however, that the framework and ethnographic research will enable a policy narrative about empowering social policy? that has the potential to move the policy framework away from programmatic bundling to a more genuinely cross-sectoral perspective on reducing poverty.
- By adding depth and a “human face” to the numbers and the relatively brief accompanying narrative of the CSCs, the ethnographic research powerfully appealed to policy makers’ imaginations and emotions. For many Jamaican policy makers, these qualitative insights and their manner of delivery were very new. The experience of delivering PEER findings made it clear that this type of analysis, with compelling accounts of young people‘s everyday experiences, has the potential to shift policy makers‘ subjective world views in a way that numbers, diagrams, and graphs cannot.
