[Boston] In June, tech company Babylon said its AI had demonstrated the ability to provide health advice that was "on-par with practicing clinicians," and it didn’t go down well.

It claimed its algorithm performed better than the average candidate in an MRCGP assessment – the final test for trainee GPs - but Professor Martin Marshall, Vice Chair of the Royal College of GPs, said:

“Computers are computers, and GPs are highly-trained medical professionals: the two can't be compared and the former may support but will never replace the latter.”

A group of researchers from the Massachusetts Institute of Technology (MIT) set out to identify what impact doctors’ "gut feelings" have when determining what tests they will order for patients.

From symptoms to family history, to lifestyle habits, a variety of factors are involved in the decision-making process, but MIT computer scientists looked at the role of intuition in clinical practice, seeking to discover whether it was "measurable."

Using a collection of medical records from 60,000 intensive-care-unit patients admitted to the Beth Israel Deaconess Medical Center in Boston, Massachusetts, over a ten-year period, the scientists performed sentiment analysis – also known as opinion mining – on doctor’s written notes.

Sentiment analysis is widely used in non-healthcare settings to determine consumer attitudes, and is based on computer algorithms that examine language and tally positive or negative sentiments associated with words being used.

The researchers sought out to determine whether more information was available in doctors’ written notes rather than in the data available in medical records, and they computed sentiment scores to see if there was a correlation with how many diagnostic imaging tests had been ordered for patients.

MIT says that, if medical data alone was leading doctors’ decisions, intuition would not play a role in their decisions.

But their study, led by Mohammad Ghassemi, a research affiliate at MIT’s Institute for Medical Engineering and Science, and computer science graduate student Tuka Alhanai, shows that when doctors felt more pessimistic about a patient’s condition, they ordered more exams, and when they felt extremely negative about it, they ordered fewer tests. The association was strongest on the first day of a patient being admitted to hospital, decreasing gradually over time. 

“As a proxy for decision-making judgment, sentiment analysis has significant potential to assess utilization of other resources, such as laboratory tests, use of ancillary services, and discharge/transfer decisions. It may also help explain variance in other areas of clinical practice, including medication dosing, and decisions to withdraw care,” the authors of the study explained. 

MIT reports that the researchers now want to establish what factors contribute to doctors’ "gut feelings" to see if AI systems could potentially learn to incorporate the same data in the future, and Ghassemi told BJ-HC: 

"We believe that, with the right data and methods, it should be possible to train machines to infer clinical sentiment, and incorporate this sentiment as part of their patient assessments.

"Close collaborations with clinicians that are interested in working at this frontier will be needed to understand how to best train and deploy systems that can best learn and use clinical sentiment."