EHR outage takes down federal Cerner systems

Clinicians at dozens of Defense Department, Coast Guard and Veterans Affairs sites were unable to update medical records for hours this past Wednesday.
By Kat Jercich
11:19 AM

Photo: Cerner

[Ed. note: This piece has been updated to include comments from the VA.]

Dozens of federal sites experienced an electronic health records outage this past week, leading to nearly three hours in which clinicians could not update medical information.  

According to Nextgov's Aaron Boyd, Oracle databases powering Cerner's systems at Department of Defense, Coast Guard and Veterans Affairs sites went offline on Wednesday evening around 5 p.m. ET.

"Cerner remains steadfast in its support of our federal clients' efforts to provide timely, high-quality care through a single, common electronic health record," said Pat Sargent, general manager of Cerner Government Services, in a statement to Healthcare IT News.   

"We continue to work hand-in-hand with our federal partners to address any and all concerns and are committed to getting this right for service members, veterans and their families," Sargent added. "Any downtime, however brief, is unacceptable, and we work each day to ensure we are delivering the best possible solutions."  

"Federal clients of the new Cerner electronic health record system (VA, Department of Defense and U.S. Coast Guard) experienced a system outage from 4:09 p.m. to 6:15 p.m. CT on April 6, 2022," said VA representatives to Healthcare IT News after publication. "The system was not accessible to end users during this time and sites entered downtime procedures, include going to paper charting until the system was turned back on.

"The incident began with a failure of one of three Oracle database instances. The root cause is not determined at this time, but specific Oracle items are being investigated," the representatives added. "All database, application and Citrix nodes were restarted, restoring service. There was no data loss or data corruption.

"Users logged back on and were able to use the system normally. A full incident report will be produced after normal major incident processes," they continued.

Requests for comment to DOD were not returned by press time.  

WHY IT MATTERS  

As reported by Boyd, the outage affected the EHR systems at 66 DOD facilities, 109 Coast Guard facilities and three at the VA.  

Overall, more than 95,000 clinician users felt the effects, although officials told Boyd there was no evidence of patient harm.  

While the main EHR was down, clinicians could review patient data through a "read-only" system, Boyd reported, but could not update any information.  

VA EHRM Integration Office Program Executive Director Terry Adirim told members of the press Thursday that staff continued with most clinical operations during the outage, relying on pen and paper to record patient information and then updating the EHR later.  

Federal News Network's Jory Heckman noted that Adirim said an investigation is ongoing regarding the cause, but pointed to an Oracle server bug.  

The systems were back online by 8 p.m. ET Wednesday.  

THE LARGER TREND

The hiccup followed a widely publicized outage at the beginning of March, in which VA staff at Spokane, Washington, and Columbus, Ohio, sites relied on downtime procedures for nearly 24 hours.  

More than 200 veteran records may have been affected by the issue, spokespeople told Healthcare IT News.  

A few weeks later, U.S. Senator Patty Murray, D-Wash., issued a statement asking VA to stop its planned March 26 rollout of the Cerner EHR in Walla Walla.  

"Just this month, I made clear to VA that if they were not ready to continue the rollout of the Cerner EHR, they needed to stop the train in its tracks. It is time to stop the train," she said in a statement.  

That launch went forward as planned, however.  

"We are excited for this new journey and have worked hard for some time to be prepared for this momentous event," said VA officials in a blog post.  

Meanwhile, the Coast Guard and DOD have continued to progress with their own EHR modernization projects.  

ON THE RECORD  

"We are in close contact with Cerner, and we are exercising all of our authority to make sure that we can position ourselves for success and assure that when these types of things happen, we get to the root cause," said Deputy VA Secretary Donald Remy in a press call Thursday, as Heckman reported.  

"We conduct a root cause analysis, we identify how the problem arose, and we make sure that we don’t see the same thing happen in the future."

Kat Jercich is senior editor of Healthcare IT News.
Twitter: @kjercich
Email: kjercich@himss.org
Healthcare IT News is a HIMSS Media publication.

Want to get more stories like this one? Get daily news updates from Healthcare IT News.
Your subscription has been saved.
Something went wrong. Please try again.