Online symptom-checkers are often wrong

By Lisa Rapaport

(Reuters Health) - Online symptom checkers often misdiagnose patients' problems, many times encouraging people to seek care for minor issues that don't need immediate attention and other times incorrectly telling people with true emergencies that treatment can wait, a U.K. study suggests.

Researchers tested 23 online and mobile apps used by millions of people who are trying to find out if their symptoms are serious and what might make them feel better. The apps were imperfect at best, offering the correct diagnosis on the first try only about a third of the time.

For triage, the apps were too cautious in situations requiring only self-care: only 33% of the time, on average, were patients appropriately advised not to see a doctor.

At the other extreme, symptom checkers typically missed the severity of the situation in 20% of cases requiring emergency treatment.

Overall, the computer programs offered accurate triage advice for 57% of the standardized scenarios that were used in the researchers' tests.

"The risk is that people will be told to get care when they didn't need it and bear the costs and inconvenience, or they will be told not to seek care when they have a life-threatening problem," said senior author Dr. Ateev Mehrotra, a health policy researcher at Beth Israel Deaconess Medical Center and Harvard Medical School in Boston, in email to Reuters Health.

Because patients may not get much useful information from a long list of possible diagnoses, the researchers rated the symptom-checkers based on whether the programs spit out the right answer first, or somewhere lower down on a list of up to 20 possible alternative diagnoses.

The software listed the right diagnosis first in 24% of emergencies on average, and for 40% of non-urgent cases. Accuracy was better for common than for rare diagnoses.

The app that did best at giving the correct diagnosis on the first try was DocResponse.com, at 50%.

Accuracy was better for the systems that used Schmitt or Thompson nurse triage protocols, standard diagnostic tools used by clinicians to provide advice over the phone. With these nurse triage protocols, 72% of patients got the appropriate guidance on the urgency of their situation, compared with 55% without the protocols.

One limitation of the study is that it used specific clinical language to describe medical conditions in the test vignettes, which may not provide an accurate reflection of how the symptom checkers would perform for patients using nonclinical terms to describe their conditions, the authors acknowledged in a report online July 8 in BMJ.

More research is needed to help build more accurate, safer and more effective tools, and symptom checkers are unlikely to completely replace the role of humans in diagnosis, particularly for patients without Internet access, said Jeremy Wyatt, an e-health researcher at the Leeds Institute of Health Sciences in the U.K., in an editorial.

SOURCE: http://bit.ly/1CrT3hm

BMJ 2015.

(c) Copyright Thomson Reuters 2015. Click For Restrictions - http://about.reuters.com/fulllegal.asp