In production machine learning systems, the model itself is only a small part of the implementation. The "scaffolding" required to run machine learning models, also called machine learning operations or MLOps, is another crucial component.

This scaffolding includes configuration, data collection, feature extraction, data verification, machine resource management, analysis tools, process management tools, serving infrastructure and monitoring, explained Dr. Yindalon Aphinyanaphongs, assistant professor for the departments of population health and medicine in the division of general internal medicine at NYU Langone Health.

How the cloud helps machine learning

"A common way of deploying machine learning-based models is to use infrastructure-as-a-service resources," he noted. "Several cloud providers, such as Microsoft Azure and Amazon Sagemaker, offer solutions that help to create the scaffolding to deploy machine learning-based models and also provide many MLOps tools."

Apart from coordinating the nuts and bolts of the model itself, one challenge is getting data in and out of the model and into an electronic health record.

"It took us about seven weeks to deploy a COVID-19-favorable outcome model in the Epic cognitive computing platform, which has been running passively in the background since May without any hiccups."

Dr. Yindalon Aphinyanaphongs, NYU Langone Health

Common standards such as FHIR and HL7 can stream input data to these external services, but the exact specifications for building the data feeds, especially with HL7, can be tedious, Aphinyanaphongs said.

"Another challenge is model sharing, which is why NYU Langone Health wanted to align with Epic in its vision of the future of cognitive computing," he explained. "Epic envisions model sharing as being as simple as possible, and easily transferable between customer sites.

"Epic's cognitive computing platform is a turnkey solution toward model deployment that integrates directly with the EHR," he continued. 

"The platform addresses the needs of data collection, feature extraction, machine resource management, serving infrastructure and monitoring. It also has just enough features that we don't have to use resources to manage the infrastructure for deploying our machine learning models."

Getting data out of the EHR

The platform addressed the aforementioned challenges. First, by specifying columns in reporting workbench reports, NYU Langone Health staff can get specific data out of the EHR in real time with a fairly broad set of transformations.

Second, staff were able to share the model with a partner using the Turbocharger package that came with it, and they were able to install the model with the full infrastructure in a matter of hours.

"Epic's platform is ideal in that it is completely passive and happens in the background," Aphinyanaphongs said. "Our organization is an Epic cognitive-computing-platform-first organization, and we always consider whether we can deploy the model in the infrastructure we have first before considering external deployment solutions."

Not every data point is available in Epic, and staff has integrations with multiple vendors where its data feeds require additional solutions, he added.

A COVID-19 favorable outcome model

"It took us about seven weeks to deploy a COVID-19-favorable outcome model in the Epic cognitive computing platform, which has been running passively in the background since May without any hiccups," he noted. "We also have shared the model package with two other Epic customers who are in various phases of champion-finding and deployment."

Aphinyanaphongs offers his peers some advice about adopting machine learning and cognitive computing technologies.

"Other organizations should carefully evaluate how they want to spend their data science resources," he advised. 

"For myself and my team, we want to focus exclusively on proof-of-concepts and building models to solve problems at our institution, rather than spending time on infrastructure and scaffolding. Turnkey deployment platforms allow targeted resourcing to go toward driving value rather than managing servers."

Twitter: @SiwickiHealthIT
Email the writer: bsiwicki@himss.org
Healthcare IT News is a HIMSS Media publication.