Artificial intelligence and machine learning have the power to spur enormous change in the healthcare industry. 

At the same time, experts caution that it could pose a threat to the privacy of patient data – as well as possibly reproducing bias and inequity.

"We know the application of artificial intelligence has tremendous potential as a tool for improving safety standards, creating robust clinical decision support systems and helping in establishing a fair clinical governance system," said Muhammad Babur, IT-program manager at Mayo Clinic.  

Still, Babur said, "Healthcare organizations need to have an adequate governance structure around AI applications" in order to safeguard patient data and ensure equitable results.  

Babur, who will be presenting at HIMSS21 in August, noted that AI algorithms depend on large amounts of data from various sources, including electronic health records, clinical trials, pharmacy records, readmission rates, insurance claims records and health fitness applications.  

"We cannot allow unchecked AI algorithms to access and analyze huge amounts of data at the expense of patient privacy," he said.  

Bias is another major issue: AI that relies on prejudiced information can produce discriminatory results.   

As one example, Babur pointed to a recent study that found AI models aimed at helping hospitals prioritize who should get kidney transplants discriminate against Black patients.  

"These kinds of findings pose a big ethical challenge and moral dilemma for healthcare organizations," he said.  

So how can health systems and organizations ensure they're enacting adequate guidelines?  

Babur pointed to a number of available tools, such as IBM Research's open-source library, to detect and mitigate biases in unsupervised learning algorithms.  

He also flagged the importance of having "human-in-the-loop" systems to get human recommendations and suggestions during development.  

In addition, he said, "I would say one safeguard healthcare organizations should have in place to advance the use of fair AI is investing in training and education of their staff.   

"This one safeguard will ensure to reduce the potential harm a biased AI application can do to the credibility of a healthcare system," he added.  

Babur says he hopes panel attendees will walk away from his session with a greater understanding of AI as a powerful tool.  

AI, he said, "can enhance the efficiency of care delivery and allow healthcare systems to provide quality care to a more diverse population."  

However, he added, "I would like my audience to know there are many challenges before AI can deliver on its promise and potential to healthcare."

He explained, "The full potential of AI to transform healthcare can be realized by setting robust standards for data quality and completeness, data access, [and] adequate governance; establishing the regulatory framework; and putting safeguards in place for the development of a fair and more equitable AI."  

Muhammad Babur will explain more in his HIMSS21 session, Biases in Datasets, Ethical Mindsets and Designing Fair AI. It's scheduled for Tuesday, August 10, from 2:45-3:45 p.m. in Venetian, Murano 3201A.

Kat Jercich is senior editor of Healthcare IT News.
Twitter: @kjercich
Email: kjercich@himss.org
Healthcare IT News is a HIMSS Media publication.