Disaster recovery demands smart planning
Disaster recovery traditionally hasn't had a very high priority in healthcare IT. Everyone knew it was important, but it came far down the list of spending priorities. That's changed significantly in the past five years.
[See also: Disasters happen: Best to be ready]
One of the things that has highlighted disaster recovery's importance is the growing use of electronic health records and the concomitant growth of networking and interconnection. If a facility loses access to its computers it loses access to EHRs, treatment plans and other vital information that directly affects patient care.
Part of the challenge is the sheer size of electronic health records. These aren't just electronic versions of scanned paper documents, but photographs, X-rays, videos and all sorts of associated data. The volume for even a small medical practice is measured in gigabytes, and much of that has to be immediately available for patient care.
[See also: Backup power system keeps data flying high]
Another problem is that some healthcare facilities are no longer responsible for their own records. Increasingly, doctors' offices and smaller providers are having their records managed in larger outside facilities, such as hospitals. When a central facility suffers a failure an entire ecosystem may be affected.
"Hospitals are becoming service providers to supply IT services to smaller facilities and physicians' offices," says Jim VanderMey, chief innovation officer for Grand Rapids, Mich.-based infrastructure developers Open Systems Technologies. "Historically, backup and data management have been driven by healthcare applications. Now we have to view disaster recovery much more holistically."
"Disaster recovery in healthcare has some interesting challenges," says Janson Hoambrecker, director of cloud disaster recovery services at San Francisco-based data backup firm EVault. "It is tough to do because you hope never to use it."
Further, like in any field, disaster recovery in healthcare is a balancing act between uptime and money.
"Everyone wants the system to always be available," says VanderMey. "But once you look at the financial requirements, then people start backing down. Disaster recovery requires a more nuanced approach."
Many hospitals now rely on third parties for some or all of their disaster recovery services. Records and other information are backed up to a remote site using the cloud or a private connection to a service which provides a secure location for data and often the ability to run the hospital's applications remotely.
In spite of these requirements, some health care facilities achieve very high uptimes. For example, Metro Health of Wyoming, Mich., manages to achieve high uptime in spite of backing up to tape in a 30-day rotation.
"Because of our proactive approach, high availability design and clustering, we achieve 99.998 uptime," says Joshua Wilda, vice president of information technology at Metro Health.
"We put more thought into high availability," he adds. "We want to be very proactive."
Still, disasters must be prepared for, planned for and practiced for.
"At Metro we practice downtimes," Wilda says. We do a bimonthly maintenance check where the team does recovery processes for data recovery and system recovery."
A common mistake is to have the attitude that "we will never go down," he says. "If they don't forward-think they'll get themselves in trouble.
"The most important thing is to insure planning to mobilize your forces quickly," Wilda adds. "The biggest downtime we have is getting the right players in place."
"You want the recovery process to be like muscle memory," he says. "You need to understand the failure points, and practice the failure points.
"Practice where you can," says Wilda. "Align heavily with safety and compliance officers. Just because it's something you don't want to think about, doesn't mean you shouldn't think about it."