C.S. Lewis argued “we can never know what might have been, but what is to come is another matter entirely.” Nevertheless, counterfactual thoughts about “what might have been” are an essential part of everyday reasoning. Intelligence analysts especially rely on counterfactual reasoning to improve future practices following any failures (i.e., to answer the question: “What should we have done differently?”) and to provide inputs to policymakers. Improving accuracy in counterfactual reasoning therefore is critical to improving many aspects of intelligence work. As part of the Intelligence Advanced Research Projects Activity’s (IARPA’s) Forecasting Counterfactuals in Uncontrolled Settings (FOCUS) program, Kairos Research has partnered with Raytheon BBN Technologies Corp to test new methods for counterfactual forecasting and to conduct research on improving “lessons learned” processes.
According to IARPA’s website for the FOCUS program, “To date there has been little in the way of research that measures the extent to which different approaches to counterfactual forecasting yield accurate vs. inaccurate counterfactual forecasts. And there is a similar paucity of research on the accuracy of lessons drawn from different lessons learned approaches. As a result, there does not exist evidence-based guidance for approaching lessons learned activities or for developing the counterfactual forecasts that are the core of such activities; and also unfortunately there is correspondingly little evidence supporting a claim that the lessons learned from current lessons learned approaches are usually the right lessons.”
Despite the glaring need for such research, efforts to improve counterfactual reasoning inevitably run up against the same stubborn fact that Lewis noted: “we can never know what might have been.” To get around this problem, the FOCUS program asks research teams to evaluate how a counterfactual would have impacted outcomes of a simulated world (for instance, the effects a particular public health policy would have had on an epidemic), and make numeric forecasts of the likelihood of a range of outcomes. “What’s unique about the FOCUS program is that the use of simulated worlds allows researchers to actually measure the accuracy of counterfactual predictions,” said Dr. Amy Summerville, a Senior Cognitive Scientist at Kairos who is overseeing experimental work for the project. “This is something that has never been attempted in counterfactual research, and thus has the potential to produce insights that will dramatically change our understanding of human cognition.”
The approach taken by Kairos and Raytheon BBN Technologies focuses on the core cognitive processes involved in counterfactual forecasting, emphasizing the use of causal models and of “outside view” analysis that considers historical “base rates” for an outcome. “One of the things that we know is a major issue in counterfactual reasoning is the potential for consistent patterns of biased reasoning. In particular, counterfactual thoughts can seriously overestimate the impact of a particularly unusual or attention-grabbing feature,” noted Dr. Summerville, an expert whose work on counterfactuals has appeared in major psychology journals and been featured by the New York Times, NPR, and the BBC. “The key to our approach is that it systematically targets these likely sources of bias, and thus should improve the ability of analysts to accurately estimate what ‘might have been’ and better anticipate what’s yet to come.”
