Underwriting, claims, and the criminal justice system: What do these three environments have in common? The answer is noise.
If you work in underwriting or claims management, you may want to read “Noise: A Flaw in Human Judgment” by Daniel Kahneman, Olivier Sibony, and Cass R. Sunstein. In this book, the authors use examples from the insurance industry to shed light on the topic of how variability in human judgments may be costing the industry millions of dollars. While Kahneman spends significantly more time on the topic of criminal sentencing than insurance in the book, this article’s focus is on the potential impact of noise in our industry.
What is “noise”? Simply stated, it is the unwanted variability in professional judgments that lead to significantly different outcomes in similar fact patterns due to the individual professional making the judgment. Noise is clearly identified in past research in criminal justice sentencing, which revealed markedly different outcomes for similar crimes where the outcome is attributable only to the specific judge assigned to the case.
Noise in Claims and Underwriting
Consider that same phenomenon in the context of insurance underwriting or claims. For example, one underwriter is asked to price a risk, and a different underwriter is asked to price the same risk. How differently would the two underwriters price the same risk? To find out, Kahneman’s group conducted a noise audit for a large insurance company that revealed as much as a 40% variation in judgments made on similar factors—with the only variable being the decision-maker. While some variability is arguably expected, accepted, and tolerated, the insurance company cited in Kahneman’s book was surprised to learn the extent of the variability.
Noise also applies to the claims environment. How much variability would you expect among claims professionals who are asked to evaluate the same bodily injury liability claim? A question to consider is whether an insurance company wants claims professionals to evaluate and settle claims on the high end, or on the more moderate end, of a range. We likely know the answer to that question.
And how do we really know where one claims professional’s judgment lies in comparison to a different professional on the same case? The financial impact becomes clear when there is too much unwanted variability.
And speaking of finances and claim outcomes, Kahneman even ventures into the abyss of nuclear jury verdicts. He tries to impart some rationale for seemingly irrational juror behavior and why nuclear jury verdicts are so unpredictable. This topic is still a mystery to most of us; however, the insurance industry would be wise to study and understand the psychological drivers of this recent phenomenon in the context of noise. Kahneman suggests one driver is the individual jurors themselves as they make decisions about the value of the claim.
A Normalized Defect?
Should noise be considered a normalized defect in the insurance ecosystem? The concept of normalized defects—as originated by NASA and then expanded into business literature—suggests that a “defect” (in this case, the unwanted variability in judgments) is basically known to the industry but ignored like the leak in the basement. So, instead of being mitigated, noise and its consequences may be seen as acceptable and tolerated, to a degree.
In the context of engineering, a normalized defect may be more easily identifiable and quantifiable after its long-standing acceptance results in a disaster. Look no further than the recent Champlain Towers collapse in Florida. Noise, of course, may not cause the insurance industry to collapse, but continually looking the other way and failing to recognize it as real is clearly a normalized defect we continue to accept despite its adverse financial impact.
Is the existence of noise in the industry, then, a potential problem? If we know that underwriting and claims are such noisy environments, should we be more aware of it, identify potential defects in the system, and take steps to mitigate the noise? Calculating the cost of noise and its impact on profits may be too onerous or deemed too expensive to address. And so, it continues to erode profits in the insurance industry.
This brings us back to Kahneman, and what he calls the “Goldilocks” premium or “Goldilocks” claim value. Specifically, what is the right settlement value of a claim, or the right premium to charge? We may never be able to predict for sure what the correct value is; and, in fact, the true correct value may never be known. But Kahneman does make some suggestions as to how to reduce the variability of judgments in general. It is up to us to apply this knowledge to our industry and use it to improve performance.
How to Mitigate Noise
Noise as a normalized defect is a topic that dovetails with recent cautions to insurance companies and TPAs that professional training could improve performance in professional judgments. Training in decision-making and negotiation, heuristics, and the biases at play in decision-making is arguably inadequate, or dismissed as not needed, or viewed as not impactful or providing a short-term measurable ROI.
Add to the lack of training the independence of decision-makers, who become complacent with their evaluations over time and go unchallenged. Experienced professionals tend to only compare their judgments to their own past judgments, which Kahneman calls “the illusion of agreement.” No one challenges the professional judgments made by the experienced professionals, but should we?
Kahneman suggests that techniques such as decision hygiene deserve a closer look. Many companies likely believe they already have high levels of robust supervision, authority levels, and checks-and-balances in place to correct or bring premium and claims evaluations in line. But then, so did the large insurance company that conducted the noise audit in Kahneman’s book. Imagine the executives’ surprise and shock when they discovered the 40% variation. Kahneman cautions that the perception that such variations apply to everyone but ourselves is one of the most important barriers to recognition and amelioration.
The Role of AI
The use of algorithms and AI to replace human judgments in areas such as claims evaluation of bodily injury liability claims has met with a lot of challenges. Even so, Kahneman suggests the use of algorithms may be helpful to reduce the variability in decisions and judgments.
Maintaining the human contact in the claims process, maintaining the dignity of the individual claimant, and maintaining the credibility and reputation of the claims professional have proved hurdles to machine learning in this environment.
But that does not mean we should not try to integrate these tools. As Kahneman points out in his book, algorithms may very well have a useful place in this process to reduce variability attributable to the decision-maker, if reduction in variability is desirable.
Looking forward, it will be optimal to find rules and standards to help reduce noise while maintaining the creativity and individuality of the decision-maker. Complex injury claims are multi-faceted, and perhaps certain decisions can be rules-based. While Kahneman does suggest the use of algorithms to reduce noise, he also acknowledges that it may be easy to get bogged down in the statistical modeling, failing to remember the personal nature of claims interactions.
At this point in time, decision-making on complex claims may be best tackled with added levels of decision hygiene, fragmented decision analysis, and increased awareness training for claims professionals on the biases that affect individual judgments.
Imagining practical applications of Kahneman’s concept of noise may be daunting—considering its causes, its impact, and the difficulty in mitigating it. At one turn, we may be convinced that algorithms, rules, and standards are a good solution. At the next turn, we are mindful that human nature is at the heart of our business, and that rules may rob us of the opportunity to be creative or use that personal touch that is often needed to finesse a desirable optimal outcome.
Realizing the heuristics and biases at play in the decisions we make every day should both give us pause and encourage us to keep looking for optimal solutions. Will the industry acknowledge the variability in our judgments and simply accept them as “the way it has always been done”—in other words, accept noise as a normalized defect? Or will we face the emerging impact and take steps to identify noise and mitigate it where possible by making the best use of human creativity and machine learning to create more desirable outcomes for individuals, businesses, and the industry?
Taking noise seriously may be a good start.