This report discusses the principles according to which one should or should not combine quantitative models and structural analogies in forecasting political instability. The goal is not to promote one method or the other, but to describe how using both methods in the most appropriate manner can yield superior forecasts.
Key findings:
- Preventing violent conflict requires early warning of likely crises so that preventive actions can be planned and taken before the onset of mass violence. For most of the post–World War II period, policymakers and intelligence agencies have relied on experts to make qualitative judgments regarding the risk of instability or violent changes in their areas of study. Such qualitative analysis generally rests on the expert’s subjective analysis of a mix of sources, including news reportage and other media sources, other open-source data, and embassy and intelligence reports. The inability of such experts to adequately predict major events has led to efforts to use social and analytical tools to create more “scientific” forecasts of political crises. A number of quantitative models, based on objective analysis of open-source data, have been offered to analysts as a supplement to their traditional analysis. Notable examples are the military’s Analyzing Complex Threats for Operations and Readiness (ACTOR) model and the Political Instability Task Force (PITF) model.
- The advent of quantitative forecasting models that give early warning of the onset of political instability offers the prospect of major advances in the accuracy of forecasting over traditional qualitative methods. While these quantitative forecasting methods should move to the foreground and play a key role in developing early warning tools, it does not mean that traditional qualitative analysis is dispensable. The best results for early warning are most likely obtained by the judicious combination of quantitative analysis based on forecasting models with qualitative analysis that rests on explicit causal relationships and precise forecasts of its own.
- When a qualitative analysis and the PITF model disagree, it is important to study both sets of predictions, and not simply disregard one or the other, because both models have different weaknesses. Qualitative analysis is likely to over predict stability because analysts tend to focus only on the kinds of problems that have arisen previously in their countries. Moreover, they tend to project the present—in which the state may seem strong—into the future. The PITF is the only forecasting model known to have a proven record of accuracy on global historical data of 80 percent or more. The PITF model may often be a better guide to problems that can lead to instability in the event of a slight shift of events. It is however prone to slightly over predict instability for intermediate regime types and to under predict instability for full democracies and full autocracies with low incomes. That is because on average, full democracies and full autocracies are the most stable regime types.
- It is important to combine qualitative and PITF models in determining the type of instability likely to arise in a given case. Policymakers and analysts should insist on a multiple-method approach, which has greater forecasting power than either the quantitative or qualitative method alone. In this way, political instability forecasting is likely to make its largest advance over earlier practices.
