78% of physicians inflate medical procedure success estimates

COVID Infection Doctor Patient

Many medical professionals are just bad at doing probability calculations.

The success of multi-step medical procedures is often overestimated by doctors.

Medical treatment is rarely straightforward, whether it’s a woman giving birth or a man undergoing a cancer biopsy. An unforeseen problem can happen at any time, especially with treatments that involve multiple steps.

However, recent research by experts at the University of Utah Health and its partners has found that doctors often have unrealistic expectations about the success of complicated medical operations. Inflated predictions of success, according to the researchers, can negatively impact treatment choices and result in unintended harm to patients.

Overall, nearly 8 out of 10 physicians who responded to the study thought there was a greater chance of getting the intended outcome of an operation than one or more stages leading to the success of that outcome.

According to Scott Aberegg, MD, an intensive care pulmonologist at the University of Utah Health, the study, which was published in open JAMA networkexposes a serious logical gap among clinicians who fail to recognize that each step of the process carries its own risks that can reduce the likelihood that the desired medical outcome will be achieved.

“Too often doctors act as if the stars align more often than they actually do,” says Aberegg. “They tend to focus on the desired outcome rather than the actual chances of success involved in each intermediate step. We cannot continue to make medical decisions like this. We need to base them on more realistic expectations.”

In order to see how often a phenomenon known as the conjunction fallacy occurs in medicine, Aberegg, Hal Arkes, Ph.D., of Ohio State University, and Kevin Arpin, Ph.D., a forensic expert at Travelers Insurance in Connecticut, conducted the research.

When a person thinks that a combination of events is more likely than any of their individual parts, this is known as the conjunction fallacy.

Suppose, for example, that a doctor notices a patient’s skin growth and has an 80% suspicion that it is cancerous. There is also an 80% chance that the pathologist will see cancer in a biopsy sample in the lab. The false assumption – the conjunction fallacy – would be that there is more than an 80% chance that the pathologist will see cancer in the patient’s biopsy sample.

In fact, the probability that the pathologist will see cancer in this patient’s biopsy is 64%, because first the patient really has to have cancer and then the pathologist has to see it in the biopsy.

“A lot of doctors are just not good at calculating probabilities,” says Aberegg. “As a result, they often miss opportunities to make better treatment decisions.”

In their study, Aberegg and colleagues asked 215 obstetricians and pulmonologists to evaluate scenarios they might encounter when caring for patients.

For example, in one scenario, obstetricians were confronted with a 29-year-old pregnant woman in labor. However, the child is not properly positioned for a vaginal delivery. In this case, doctors were asked to estimate the likelihood that the child would move into a birthing position and be born without the need for a cesarean section.

Overall, 78% of clinicians who evaluated one of the three survey scenarios estimated that the probability of the desired outcome would be greater than the probability of the two individual events required for it to occur. This is a mathematical impossibility, says Aberegg.

“Our study shows that if you misestimate the probability of two events occurring to get the desired outcome, you can put your patients at unnecessary risk,” says Aberegg. “In the case of the birth scenario, you could end up waiting a long time for that baby and end up having to have a C-section anyway. This delay can be harmful to both the mother and the child.”

All physicians who participated in the surveys had an average of 25 years of experience. However, this experience did not seem to stop them from opting for the conjunction fallacies presented in the study. However, this is not too surprising, as previous research has found that nearly 50% of medical students are prone to these types of probability errors, according to Aberegg.

“There are huge opportunities in medical education to improve the curriculum in terms of teaching the importance of probability in medical settings,” says Aberegg. “Numbers are the most reliable source of correct decisions in medicine.”

Aberegg urges physicians to not only trust their experience, but also do their best to stay up to date on the latest probability research published in medical journals on various conditions and procedures.

Among the limitations of the study is that participants were asked to respond in writing that they might have been different if they were seeing real patients.

However, Aberegg believes the study could have broad implications.

“Our results are very strong,” says Aberegg. “We are confident that they represent a widespread phenomenon in medicine. I am interested in cataloging more examples so that the full breadth of this potential problem can be exposed and hopefully resolved.”

Reference: “Analysis of physicians’ probability estimates of a medical outcome based on a sequence of events” by Hal R. Arkes, Ph.D., Scott K. Aberegg, MD, MPH, and Kevin A. Arpin, Ph. D. , June 27, 2022, Open JAMA network.
DOI: 10.1001/jamannetworkopen.2022.18804

The study was self-funded by Aberegg, Arkes and Aprin.


#physicians #inflate #medical #procedure #success #estimates

Leave a Reply