Q&A: How redundancy enabled resilience after Crowdstrike outage
Photo: CHI Health
When CrowdStrike tried to parse a bad Falcon Suite configuration file pushed to millions of Windows machines this past July, the widespread crash took out the whole operating system and its applications – affecting countless machines across numerous industries around the world.
The global IT outage caused widespread disruption of healthcare delivery, with many hospitals and health systems losing access to electronic health records and other mission-critical patient-care systems, and being forced to return to using paper.
Omaha, Nebraska-based CHI Health was one of them.
But Dr. Abhishek Singh, a neurologist at CHI's Creighton University Medical Center-Bergan Mercy, said cloud-based analytics helped maintain his ability to care for patients in the immediate fallout from the Crowdstrike crisis.
Using the mobile-based imaging-analysis tool RapidAI, Singh says he was able to gather intelligence on a stroke patient from his home and lead his team remotely in treating the life-threatening condition.
For Singh, the ability to access mobile-based imaging analytics "created a kind of contingency system for us during a time when our laptops and our brick-and-mortar systems were failing," he tells Healthcare IT News.
For the healthcare industry, network outages – whether caused by cyberattacks or other types of network outages – can put patients at risk of harm, making redundancy a critical component of business continuity planning.
The Crowdstrike incident was an object lesson in how "growing reliance on interconnected IT systems has expanded the risk surface," as House Homeland Security Committee Dr. Mark Green, R-Tenn., said this past month.
The outage demonstrated "the urgency of promoting cyber hygiene and resiliency," said Green, as the committee called Adam Meyers, senior vice president of counter-adversary operations at CrowdStrike, to testify on Sept. 24 before its Subcommittee on Cybersecurity and Infrastructure Protection.
Meanwhile, two months after the event Microsoft announced that the Crowdstrike outage would be a top agenda item of discussion at the closed-door Windows Endpoint Security Ecosystem Summit happening September 10 at its offices in Redmond, Virginia.
We spoke recently with Dr. Singh of CHI Health about his experience maintaining care continuity in the wake of the Crowdstrike incident.
Q. What happened to CHI Health’s clinical workflows during the Crowdstrike outage?
A. Like many other sectors – airlines, businesses, local governments and more – when this security patch from Crowdstrike was implemented in our health system, it, unfortunately, led to a temporary loss of access to our computer systems as they had to be shut down.
Although our IT team immediately began working to restore functionality, the outage disrupted our EHR system, communication tools and any operations that depended on the affected computers or laptops. Over the past 10-15 years, our healthcare system has transitioned from paper charts to electronic records, which we have become extremely reliant on.
Of course, despite the outage, patients did not stop needing care.
That morning a critical situation arose when a young patient arrived directly at our hospital’s ER in serious condition, suffering from a posterior circulation stroke. This type of stroke affects a part of the brain that can lead to high mortality and morbidity, and we typically lose four out of five of these patients.
In stroke care, specifically, our ability to view brain images is often entirely dependent on computer systems and as a comprehensive stroke center, it is crucial for us to have immediate access to imaging and to maintain the ability to communicate amongst teams.
With our documentation and many of our communication systems down, our radiologists, neurologists and other doctors faced significant challenges in providing care. From the moment the patient arrived, we were at a disadvantage without access to all the imaging we typically have. Fortunately, we were able to manually review the CAT scan – in my case, from outside the hospital – using the mobile-based RapidAI platform.
This showed that the patient had no bleeding and confirmed that we needed to administer the IV clot-busting medication Tenecteplase.
I was also able to use the mobile tool to communicate with the care team who administered the medication in time and activated our neuro-interventional radiology team to perform an intra-arterial treatment. As a result, the patient showed remarkable improvement and was discharged three days later.
Q. How did access to imaging analysis and clinical workflow technology provide an avenue to maintaining patient care during the Crowdstrike outage?
A. Having access to this type of technology during the tech outage and system failure was nothing short of critical, because it created a parallel system that delivered imaging analysis and clinical-workflow capabilities via our mobile phones.
It gave us the critical and time-sensitive information that we needed to maintain patient care. Having these redundancies, plus flexible cloud-based technology, allowed us to maintain continuity of care during an outage of this particular nature.
Q. What was the ultimate value to patients in the aftermath of the BSOD crisis?
A. In this particular situation, having access to mobile analytics may have saved the patient’s life, but every day these advanced tools play a vital role in reducing miscommunication and improving the speed and accuracy of diagnoses.
This time sensitivity is especially critical in stroke cases, because every minute without care results in billions of neurons lost that can never be restored. We as physicians need to be able to quickly determine whether or not there is bleeding, clots, their location or size, and the best treatment path.
What makes some of these tools even more valuable is the ability for specialists like myself to use them from outside the hospital when we’re on call or potentially covering another hospital in our network because of the shortage of specialists.
This is particularly common in the Midwest and other rural areas, where specialized expertise is just less common.
Fortunately, artificial intelligence tools help us extend and share our knowledge with more people and fill gaps across our health systems. By securely accessing patient information and making informed care decisions from off-site, we can help ensure patients are not left waiting so long for our treatment guidance.
Q. How has AI helped improve the diagnoses of vascular and neurovascular conditions?
A. In a nutshell, over the last 10-15 years, AI algorithms have greatly improved their capacity for diagnostic accuracy and impacted the timeliness of diagnoses.
For example, when RapidAI received U.S. Food and Drug Administration approval to analyze the presence of blood in a CAT scan, it was crucial because clot-busting treatments can’t be administered to patients with brain bleeds.
It was the same when the tool gained approval for accurately measuring intracranial hemorrhage volume because previously this was done manually using a 40-year-old formula, which was time-consuming and less precise. Now, deep clinical AI automates this process for us and provides precise measurements of bleed volume, which are vital in clinical practice and serve as an excellent tool for benchmarking and assessing patient outcomes.
AI tools also give us more to work with, which is extremely valuable.
Down the line, I think we’ll see even more advancements, particularly in developing better scoring systems across medicine and definitely in stroke care. Before long, AI will help us predict bleeding and complications from procedures, guide patient selection for new and emerging treatments and optimize the triage process to help us prioritize our limited resources even more efficiently and automatically than it does now.
Andrea Fox is senior editor of Healthcare IT News.
Email: afox@himss.org
Healthcare IT News is a HIMSS Media publication.
The HIMSS Healthcare Cybersecurity Forum is scheduled to take place October 31-November 1 in Washington, D.C. Learn more and register.