It is widely accepted that evaluation is a social process which implies the need for a participatory approach. But what is understood by ‘participation’? This critical review from Hull University Business School argues that the blanket use of the term has masked the heterogeneity evident in its realisation in practice and highlights a lack of transparency in participatory methods in evaluation.
Four key arguments are typically advanced to support the notion of participation in decision-making, design and planning: ethics, expediency, expert knowledge and motivating force. Yet the notion of participation is ill-understood and is an important problem across a range of methodologies in evaluation. Participation should be explicitly considered rather than ignored or implicitly assumed. In this sense, the systems field may offer guidance on how best to realise a participatory approach. The problem of participation can only be approached through an understanding of power and its realisation in practices that prohibit or promote participation.
The lack of evident success in practice is a result of a general failure to make the nature and aims of participation explicit. To what extent are evaluation methodologies guilty of this?
- Rebien’s ‘Continuum and Criteria’ are based on different degrees of participation. He advances a continuum of participation with a threshold to identify ‘truly’ participatory projects. Yet Rebien’s criteria for distinguishing participatory methodologies are insufficiently defined and may promote practices which impact negatively on participation.
- Guba and Lincoln’s ‘Fourth Generation Evaluation’ is analysed through Oakley’s obstacles to participation. Whilst explicitly participative in nature, it includes some rather naïve assumptions which are unlikely to promote participation.
- Patton’s ‘Utilisation-Focused Evaluation’, grounded in realism, advocates limited composition and size of the evaluation task force. Patton’s ideas about methodological transparency and capacity building aspirations are unrealistic and his approach may result in only limited participation.
- Pawson and Tilley’s ‘Realistic Evaluation’ seeks to explain how mechanisms operate in contexts to produce outcomes. It is based on the separation of roles and on a limited form of participation. Consequently, the knowledge produced is critically restricted.
- Taket and White’s ‘Working with Heterogeneity’ stems from systems analysis. They advocate a form of evaluation based on ‘pragmatic pluralism’. However, their approach may promote an expert driven form of evaluation. Whilst acknowledging the influence of power on participation, their recommendations for dealing with power are not well developed.
Only through an appreciation of power can the problem of participation be addressed. Participatory methodologies must be simple if they are to be transparent and easily transferable. As a starting point, Ledwith’s ‘Sites of Oppression’ matrix is recommended as a good way of exploring how the processes of power operate:
- The matrix illustrates the potential ways in which oppression overlays and interlinks. It includes elements of oppression and the different, reinforcing and interdependent levels at which oppression operates. It facilitates a deeper analysis and offers a wider perspective.
- Such an analysis would benefit those of a pluralist orientation by clarifying who should be involved in the evaluation, what barriers exist to prevent participation, and how these might be removed.
- Those with a more realist bent would gain an appreciation of how mechanisms/structures work at different levels. This analysis would draw on the propositional, experiential and practical knowledge of the subjects of evaluation.
