The Coalition for Health AI has announced the availability of the open-source version of its artificial intelligence Applied Model Card on GitHub. The healthcare industry-led coalition said Thursday it designed the card to enable healthcare AI developers to provide critical information about how their AI systems are trained.

According to CHAI CEO Brian Anderson, sections of the draft open-source model card – a healthcare AI 'nutrition label' – go above and beyond the U.S. Health and Human Service's Health Data, Technology, and Interoperability: Certification Program Updates, Algorithm Transparency, and Information Sharing Final Rule for certifying health IT systems. 

The nutrition label also offers opportunities to align with other voluntary standards, such as the National Academy of Medicine's AI code of conduct.

"It's an important step in starting the conversation between a customer and a vendor rather than just leaving it up to a PowerPoint slide and anecdotal stories like to build trust," Anderson told Healthcare IT News on Wednesday.

Built by consensus

Driven by growing demand from both startups and health systems, CHAI has said its mission is to ensure that anyone building and using AI in health can make informed decisions and the open availability of its nutrition label offers greater transparency and trust in their selected AI tools. 

If we want doctors, nurses and patients "to trust AI models that are going to be used in more and more consequential use cases in health, we need to have greater transparency about how these models were made and how they perform," he said. "The model card gets to that kind of transparency."

Any HIT company or health system can use CHAI’s model card in any way they want, which the coalition said could help streamline the procurement processes and improve implementation at scale.

"We want the model card to be widely available and widely used by the vendor community and by the customer community," Anderson said.

The CHAI nutrition label was created from a concerted multi-stakeholder effort to build "a consensus set of definitions about what responsible AI looks like," Anderson explained. This includes the agreed-upon metrics for evaluation, how to think about performance and assessments of fairness and bias.

The coalition has sought to unite regulators and developers to drive AI standards.

"It is a real challenge when you bring together vendors, developers of AI models and their customers and try to build consensus around what is the minimum level of transparency that we can agree to that we need from the developers," he said of the work of the last eight months.

While CHAI said in October that the certification rubric and model card designs can be expected after incorporating stakeholder feedback by the end of April 2025, the coalition is asking for feedback from testing via GitHub to come in on or before January 22.

Standards and alignment

Though the organization has nearly 3,000 member organizations, making the nutrition label freely available ultimately helps health systems have hundreds if not thousands of AI tools. A digital open-source coded version of the model card is a standard that can be used repeatedly.

"You want to have scalable solutions that can rise to the challenge of managing and monitoring" multiple AI systems and tools, Anderson said.

Another thing that Anderson spends a lot of time thinking about is alignment.

For example, many in the industry are focused on patients being part of the development process. 

"We think that's really important because our patient community groups believe, and I believe if we want to put patients in the center of this, developers should do that from the beginning" in a way "that meets patients where they're at," 

The CHAI AI model Card also includes a section around the National Academy of Medicine's AI code of conduct, which is not part of the HTI one rule. 

"We believe that making sure that vendors have the opportunity to articulate, if they believe that their model's development is in alignment with the AI code of conduct that NAM describes, that that should be there, and that they should have the opportunity to share their perspective on if they believe the models developed in accordance to that."

Andrea Fox is senior editor of Healthcare IT News.
Email: afox@himss.org
Healthcare IT News is a HIMSS Media publication.