Large language models are fast becoming a key building block in new information systems for administrative staff and clinicians at hospitals and health systems. This form of artificial intelligence can accomplish tasks no human being can imagine doing.

Harrison.ai develops artificial intelligence technology to speed clinical diagnosis and offers a suite of AI radiology and pathology tools. They're designed to improve efficiencies for physicians to help address burnout.

Healthcare IT News spoke with Dr. Aengus Tran, cofounder and CEO of Harrison.ai, to talk about LLMs and radiology: why they're a good fit, what genAI models can do for a radiologists, how a radiologist can be assured of their quality and accuracy – and how adoption of LLMs for radiology can help address the radiologist shortage.

Q. Why is a large language model a good fit for radiology?

A. Large language models have the potential to address some of radiology's most pressing challenges. While many of the AI models that have made inroads in healthcare are only capable of predefined tasks, advancements in machine learning are enhancing new models' ability to undergo continuous learning and generalize to areas in which the model has not been trained.

This is a transformative next step for AI in healthcare – an industry where drawing conclusions based on previous experience and knowledge in the face of new and unknown conditions is critical to providing the right care for patients.

The way that radiology LLMs are trained is not dissimilar to how medical students learn diagnostic radiology – through constant practice, reviewing of cases and studying of literature. A well-trained LLM model should be able to achieve human-level performance on tasks like parsing radiology images to detect anomalies, localizations, comparing to priors and predicting outcomes.

LLMs could have an immediate and direct benefit for radiologists, as it supports them in addressing the rapid expansion of medical data by swiftly processing and integrating information from multiple sources.

Whether interpreting textual data like medical literature and patient histories or analyzing visual imaging data, these models can provide radiologists with comprehensive insights that previously demanded considerable time and resources to compile.

Additionally, because radiology images are digitized, there is a wealth of high-quality, standardized data that is unique to the field and ripe for AI intervention.

Q. What can an LLM do for a radiologist?

A. Medical facilities worldwide are grappling with increasing volumes of medical images and associated data per case, a shortage of radiologists, and the risk of physician burnout.

A radiology-specific LLM could rapidly process medical information, patient histories and imaging data, potentially offering radiologists comprehensive insights in a fraction of the time.

Additionally, LLMs could aid with diagnostic decision support for radiologists by interpreting imaging data, identifying anomalies, suggesting possible diagnoses and automating time-consuming administrative tasks. Radiologists then can make quicker and more accurate decisions, allowing them to see more patients while reducing their overall workload.

Contrary to early concerns about AI replacing radiology jobs, LLMs – or at least how we see them developing – are not intended to replace human expertise, but rather to enhance and augment it.

While many of the LLMs out in the world are powerful, they have a broad, generic focus.

These generalist models are not suited to a field that's utterly dependent on accuracy and cannot accept errors. A specialized and highly nuanced function like healthcare requires a specialized model.

Q. How can a radiologist be assured of the quality and accuracy of the work an LLM is doing for them? How can they be comfortable?

A. A model is only as good as the data that it is trained on – and we need to be sensitive to the risks and challenges associated with the use of LLMs. The effectiveness of LLMs hinges on three key elements of their training data: quality, volume and diversity. By leveraging datasets that excel in these aspects, we can create sophisticated systems capable of generating precise and high-quality outputs.

Furthermore, comprehensive evaluation is essential. Evaluating LLMs for use in radiology comes with added challenges – to evaluate foundational models, we must move to a paradigm where we test them on their abilities to recognize individual pathologies and their radiology interpretation skills in general.

What this means is there must be even more stringent tests for safety and accuracy for LLMs. This involves testing against international standards and benchmarks, comparing performance across other LLMs in the industry, and subjecting the models to real-world assessments.

Several benchmarks have been introduced to evaluate and compare the performance of multimodal foundational models on medical tasks. Our view is LLMs should not only be tested against these benchmarks, but also against exams taken by radiologists, who are considered the gold standard when it comes to interpreting medical images.

This rigorous evaluation process serves a dual purpose: It builds confidence among radiologists by demonstrating thorough validation of the model while simultaneously establishing its legitimacy as a reliable assistive technology.

Q. How can adoption of LLMs for radiology help address the radiologist shortage?

A. Global healthcare is facing multiple intersecting challenges, including rising imaging volumes and associated data per case, a shortage of medical professionals, and risk of burnout for the remaining staff. LLMs can potentially help to address these issues by enhancing productivity and efficiency in diagnostic processes:

• They can improve manual data annotation efficiency to create large, labeled datasets for comprehensive medical imaging AI.

• They can allow for easy access and retrieval of cases by parsing radiology reports, thus facilitating fast, efficient and continuous quality assessment.

• Importantly, as a model that can work anywhere, at any time of the day, LLMs can facilitate better access to radiology services in underserved and remote areas. This can mean providing preliminary readings and support for clinicians who may be working in isolated locations or in facilities with limited resources, improving equitable access to timely and accurate diagnoses for patients around the world.

Most of these are time-consuming activities that can be streamlined by AI, allowing radiologists to focus on the critical decision-making elements of their work that have the highest impact on patient care.

Follow Bill's HIT coverage on LinkedIn: Bill Siwicki
Email him: bsiwicki@himss.org
Healthcare IT News is a HIMSS Media publication.

The HIMSS AI in Healthcare Forum is scheduled to take place September 5-6 in Boston. Learn more and register.