Moral and ethical decision making often involves choices where a decision maker is forced to choose among multiple imperfect options. For instance, doctors performing triage after a disaster must decide which victims receive care- and thus, which will not. Recently published research from Kairos Research Senior Cognitive Scientist Dr. Amy Summerville and intern Brielle Johnson investigated the emotions associated with these moral decisions. In this research, Summerville, Johnson, and a team from Miami University and Florida State University investigated whether decisions that minimize overall harm (known as utilitarian decisions) may evoke more feelings of regret even when the decision maker does not think that the choice should have been different, as compared to choices that avoid direct harm at a greater indirect cost (known as deontological decisions). By separating out the emotional and cognitive aspects of regret, this research allowed new insight into the forces that shape reactions to moral decisions. For instance, asking decision makers to imagine if they had made the opposite choice reduced feelings of regret for utilitarian but not deontological decision makers.
While the focus of this research was on human rather than machine cognition, according to Dr. Summerville: “Understanding the architecture of human moral decision making is crucial to ensuring that the design of future AI systems is aligned with human moral and ethical norms.” Dr. Summerville further noted that, “As AI takes on even greater autonomy, one of the biggest challenges programmers will face is not what it can do but what it should do. This research highlights the fact that our cold, intellectual understanding of a decision can diverge substantially from how we feel about that decision.”
The paper appears in the September 2020 issue of Personality and Social Psychology Bulletin. A full text preprint is available here.
