Quantitative and qualitative methods of research each have strengths and weaknesses when applied in isolation. However, combining the two approaches through mixed-method evaluation is gaining wider acceptance among social science researchers as a way of conducting more comprehensive and robust analysis. This chapter from RealWorld Evaluation: Working Under Budget, Time, Data and Political Constraints discusses the most appropriate contexts and strategies for using a mixed-method approach. It argues that mixed-method evaluation is a flexible and practical technique which can be used at any stage of an evaluation. Nevertheless, a fully integrated approach requires extensive planning and deliberation to ensure that the most appropriate combination of methods is chosen and successfully implemented.
Mixed-method evaluation combines the detailed insights and holistic understanding obtained from qualitative research with the ability to generalise to a wider population offered by quantitative data collection. Thus, it allows for a more comprehensive analysis. Mixed-method designs can be employed to strengthen validity, fine-tune sampling and instrumentation, extend the coverage of findings, conduct multi-level analysis and generate new and diverse insights:
- Mixed-method evaluation designs can give equal weight to qualitative and quantitative approaches, or allow one method to figure more prominently than the other.
- The different methods can be used concurrently or sequentially. Sequential designs are often easier to organise, although they may require more time if the second phase of research cannot begin until the first is completed. Concurrent designs can be logistically more difficult to manage, particularly if evaluation teams lack extensive experience of coordinating quantitative and qualitative methods simultaneously.
- Mixed-methods can be fully integrated throughout the evaluation process or used at any individual stage of the evaluation. However, in practice, mixed methods are generally applied at only one or two stages, be it hypothesis formulation, data collection, analysis and follow-up, or presentation and dissemination of findings.
- Mixed-methods permit multi-level analysis through the comparison of findings from data collected at the level of the individual household, group, organisation or community
In practice, undertaking a mixed-methods design has important implications for the planning and implementation of the evaluation. The benefits of mixed-method approaches vary depending on the specific weighting and application of the combination selected. The objectives of the project, along with resource constraints, will dictate the nature of the most useful combination of methods:
- Concurrent designs may save time overall, but require a large amount of planning and organisation compared to sequential approaches.
- While some evaluators refer to the use of a mixed-method design when they have only included additional data collection methods to a dominantly quantitative or qualitative approach, this is a misunderstanding of the approach. A mixed-method approach requires an integrated strategy in which the strengths and limitations of quantitative and qualitative methods are recognised and the evaluation is designed to take advantage of the complementarities between the different methods.
- A fully-integrated approach requires ensuring an interdisciplinary perspective at all stages of the research, including: composition of the research team, designing the evaluation framework, data collection, and analysis and follow-up.
- Fully-integrated approaches will generally require more time. Where research teams involve professionals from multiple disciplines, it is essential to invest additional time during the planning stage of the evaluation for building relations and common understandings among team members.
- One ongoing challenge is to develop sampling procedures that ensure subjects and cases for quantitative and qualitative data collection are drawn from the same universe. Frequently, it is difficult to know whether differences in the findings reflect real differences in the information obtained or are partly due to the different selection procedures used in each part of the study.
