This report identifies empirical evidence on the outcomes of security and justice (S&J) interventions, and provides a visual overview of its findings in an evidence gap map. The evidence gaps appear where outputs and outcomes are less tangible and more difficult to operationalise and measure. The evidence is scarce for a number of justice-specific output and outcome categories, but stronger evidence can be found on more general outcomes and outputs that apply across a wider range of intervention types.
The evidence mapping included a rigorous review of the S&J evidence base. To conduct the review, selected sources – including journal indices, online research and evaluation repositories, resource centres and experts – were interrogated. Studies that explored interventions and results were included in a database and coded according to the publication type, thematic focus, intervention, output and outcome categories. The database is also available to download. It provides a searchable list of the documents that met the inclusion criteria and enables users to generate bar charts showing the frequency of documents in each criterion.
Features of the evidence base
Research design: Ninety-five per cent of studies use observational research designs. The majority of these use a variety of qualitative techniques (interviews, focus group discussion, ethnography, case study analysis, historical analysis and political economy analysis). Only eight experimental studies were identified. These cover a range of country contexts (India, Nicaragua, Colombia, Liberia, Peru and Trinidad and Tobago) and focus on fairly specific and discrete interventions. Fifteen secondary studies were identified, largely consisting of non-systematic literature reviews and annotated bibliographies.
Publication form: The majority (49%) of documents are peer-reviewed journal articles. Thirty-five per cent of studies are classified as ‘other’ reports; they include analytical reports and case studies, and were mostly published by academic organisations, think tanks and non-governmental organisations. Few evaluations (6%) met the inclusion criteria. Those that did meet them include thematic evaluations of donor security and justice programming, synthesis programme evaluations and other individual programme or country evaluations.
Geographic scope: Western Africa is by far the most studied region. South Asia and Eastern Africa are the next most studied regions. There are very few studies exploring Central Asia, East Asia, Eastern Europe and Northern Africa.
Thematic focus: Policing is by far the most studied theme, followed by access to justice/legal empowerment, justice sector reform, legal reform and non-state actors. Few studies cover border security and intelligence. Individual studies often explore interventions that addressed more than one theme/sector.
Gap map analysis
The gap map illustrates where there is evidence. It does not provide information on what the evidence says, or on the nature of links between interventions and outputs or outcomes. It provides the basis for identifying potential areas for future research.
Interventions
- The evidence is abundant for: capacity building of organisations; strategic/statutory frameworks and legislation; community-based approaches; and restructuring of the security and justice sector. Many studies provide examples in which a combination of these interventions is used.
- The evidence is weak for: preventative interventions (integrated efforts to prevent violence and crime); capacity building for non-state justice forums and local justice facilitators; and integrated political engagement/activities that promote political will.
Outputs
- The evidence is abundant for: the capacity of state and non-state organisations to deliver; strategic frameworks; confidence, trust or satisfaction on the part of citizens in providers; and roles, coordination and dialogue amongst organisations and agencies. These are considered to be key outputs for S&J programming, they are relatively tangible, and are directly related to intervention categories.
- The evidence is weak for: application, compliance with and interpretation of laws; and state/non-state linkages. These are both justice-specific outputs.
Outcomes
- The evidence is abundant for: ownership of reforms by national and local stakeholders; resource allocation/funding stability and sustainability; access to provision; stability and outbreaks of conflict or violence; and human rights measures implemented to improve compliance. These outcomes are considered core tenets of S&J and/or are relatively tangible.
- The evidence is weak for: incentives for improved service delivery amongst security and justice actors; actual crime rates; legal awareness and confidence; gender-based violence rates; and judicial redress to protect rights. The evidence is particularly limited for longer-term development outcomes such as: economic development (local or national); poverty reduction; access to land, inheritance and property rights; access to public services and economic resources; and women’s empowerment and gender equality.