Artificial intelligence tools in healthcare, as with any other software, are not immune to bias – especially if they have been trained on data sets that do not accurately reflect the population they ostensibly serve.  

And tackling bias in AI and machine learning goes beyond recognizing its existence, said Chris Hemphill, VP of applied AI and growth at SymphonyRM.  

"Our focus is on patient engagement and getting the right communications out to the right people," they told Healthcare IT News. "But we want to make sure we're not missing people who belong to disadvantaged classes."  

At SymphonyRM, they explained, the team focuses on understanding next steps to help guide patients in engaging in personal healthcare.  

A challenge, Hemphill said, is how those next steps are determined.   

If a service focuses on need, they said, "You have to figure out what [measurement] best represents the need people have."

That might be determined by how frequently patients come in for a particular service, or what their medical spending has looked like over a certain period of time.  

"As innocuous as all of this sounds, anything you're using to evaluate need, or any clinical measure you're using, could reflect bias," Hemphill said.   

For instance, a data-driven approach based on patients' diagnostic codes may reflect the bias that led to those codes in the first place.   

Or a clinical frequency measurement might fail to take into account the hurdles – such as economic barriers or racial bias – that could have stood between a patient and seeking care.

The result can be discriminatory – and sometimes dangerous.  

At SymphonyRM, Hemphill said, their team works on ways to address approaches to patient engagement that may be falling short.

They used the example of cardiology outreach campaigns. A typical strategy, they said, is to target men over 45 and women over 55.  

"But by going with that approach – what about all the men who are below 45 or the women who are below 55?" they said.  

SymphonyRM took a machine learning approach to identifying the nuances, such as systolic blood pressure measurements, that could be indicative of cardiovascular risk factors.  

That approach resulted in a 27% growth in outreach for people between the ages of 18 and 45.  

At the same time, Hemphill noted, that initial model wasn't perfect: "It didn't perform to the level we found acceptable for Black and Asian patients."  

"A good model performance can mask what's really going on under the surface," they continued.  

For health systems and IT stakeholders who want to take steps toward correction, Hemphill recommended the paper "Ethical Machine Learning in Health Care." 

"That's a good starting point, but I want people to think beyond that paper," they continued. "There's the machine learning part, but there's also the people part.

"Modeling means nothing if you don't have the user experience; the user discussions; the training about how and why people should use it," they added.  

Hemphill pointed out that there is, of course, a moral imperative in healthcare to ensure that no patients are left behind or put at higher risks because of artificial intelligence tools.

But they also flagged the economic imperative, specifically for hospitals.  

Referring to the cardiology outreach program, they said, "If 27% of your growth came from people who are 18 to 45 across the gender spectrum, why would you exclude that?"   

"Does it take additional work, and effort, and questions to model and build in such a way?" they asked. "Yes. But I think it's well worth it."  

Kat Jercich is senior editor of Healthcare IT News.
Twitter: @kjercich
Email: kjercich@himss.org
Healthcare IT News is a HIMSS Media publication.