With artificial intelligence making its way into so many facets of healthcare, leaders are having to weigh the security, efficacy, ethics and consequences of the fast-evolving technology.

Hospitals and health systems stand to make major leaps in quality, safety, efficacy and innovation from advancements in AI and automation. But are providers properly equipped to integrate these tools responsibly?

Rob Purinton is vice president of analytics and performance improvement at AdventHealth, and leads the health system's AI Advisory Board.

The board takes a rigorous and principled approach to AI adoption and development within the Florida-based health system, gathering a cross-functional team of experts including physicians, IT experts, data scientists and the health system's vendors, including Microsoft, Vizient and Premier.

We spoke recently with Purinton to discuss why AdventHealth felt the need to create the AI Advisory Board, how it's structured, how machine learning is improving diagnostic accuracy and upholding patient safety at the health system, and what it's learned so far from building and implementing in-house AI tools.

Q. Why did you set up your AI Advisory Board? What need were you filling?

A. We decided to engage our clinical leaders on the board to help them become well-informed advocates for the tools we ultimately decide to implement within AdventHealth. There’s no shortage of hype, myths, facts, actual tools and snake-oil that are part of the national conversation on AI.

By having this conversation within a facilitated environment, we can help our clinical leaders make sense of the noise and better support a responsible path forward with AI in healthcare.

Q. What are the AI Advisory Board’s principles for vetting AI tools in healthcare?

A. The group has a first draft, one we are actively using to evaluate options for tools that address problems like sepsis, physician burnout and more. We are using a funnel approach that starts with a problem statement, allows many different AI and non-AI options into the funnel, and then systematically narrows the options to one we might want to implement.

These are the questions we think through as part of the vetting process: Is it aligned with our mission and vision? Is it feasible within our technology framework? Is it safe, ethical and sufficiently transparent to be evaluated ongoing? Does it scale to handle the volume of a large health system? What is the expected workflow benefit to our clinicians? What’s the payback period/break-even if the tool is intended to reduce costs?

Finally, what’s the real-world impact as implemented in the field? The answers to these questions are not always easy to discern, but they create higher confidence in AI tools that emerge from the funnel.

Q. What are some of the ways AI is improving diagnostic accuracy and speed of diagnosis and upholding patient safety at AdventHealth?

A. One of the ways AI is improving diagnosis is by saving our radiologists time during their daily work. For example, we use an AI tool that helps to summarize impressions on imaging reports where findings are already identified by a human as expected.

This savings of time allows radiologists to spend more time on harder exams or to dig deeper into previous exams. Another example is an AI tool that allows earlier detection of stroke, moving up the time for intervention. Earlier stroke treatment reduces death and disability.

Not all problems require AI as a solution, and it’s just as important that we not try to replace every traditional algorithm with machine learning.

Rules embedded in our Epic EHR are safeguards for safety issues like medication errors, alerting clinicians when drug interactions or allergies pose a risk. For this reason, we don’t just evaluate AI tools against each other, but also against our best non-AI alternatives, as well.

Q. How has AdventHealth gotten physicians into the process of AI vetting and implementation? Why is it important for them to have this role?

A. One of the most critical ways physicians are involved is our revamped clinical IT governance process. The committees that evaluate clinical solutions, including AI, are attended and led by our physician leaders with significant clinical practice experience.

In addition, our workgroups that vet AI tools are well-attended by physicians, who are able to ask questions directly of vendors, as well as by our data scientists. As we gather and synthesize answers to the questions in our vetting funnel, our CMOs and CNOs are able to ask follow-up questions, ask directly for clarification, or ask to have data science concepts explained.

When AI tools are implemented, we then can count on these clinical leaders to educate or even champion their use.

Q. What are some of the learnings you’ve achieved from implementing AI tools and building new in-house AI tools to solve healthcare challenges?

A. The joy of working in this area and this era of technology is that we are learning from this work every day. In implementing third-party tools, one of our learnings is that some vendors don’t have clear answers to our vetting questions at hand. There’s an expectation that a sensible workflow and positive ROI are sufficient responses.

As more elements of a standard, national “nutrition label” for AI tools are expected, we anticipate the vetting process to get easier on both the provider and AI vendor side. In building in-house AI tools, our most important learning is that the practice of putting a machine learning model into daily operation (MLOps) is as challenging as building it in the first place.

Problems of data quality, timing, processing and drift in predictive accuracy can be the result of poorly managed MLOps, as much as the AI model itself. AdventHealth data engineers and data scientists are working directly with our stack vendors like Microsoft and Snowflake to learn and improve in-house AI implementation.

Follow Bill's HIT coverage on LinkedIn: Bill Siwicki
Email him: bsiwicki@himss.org
Healthcare IT News is a HIMSS Media publication.