Kansas Dept. of Health and Environment works to restore electronic data after IT failure
The Kansas Department of Health and Environment (KDHE) is in the midst of handling the repercussions of a recent IT infrastructure failure, which coincided with an increase in record requests as students are preparing to enroll and return to school.
KDHE reported that on Aug. 5 that it began experiencing technical problems, which caused 85 percent of its servers to be affected, causing disruption in many services Kansans utilize.
"While we experienced a failure of our IT infrastructure at a time when service requirements are especially high, it's important to note that no data has been compromised or lost," said Roderick Bremby, KDHE secretary. "This outage has caused many problems for Kansans and we are very sorry for the inconvenience this has caused. Now, I can say with certainty, that KDHE is on the road to recovery."
Officials said the technical problems stemmed from a failure of the storage area network (SAN). The SAN is a central component of KDHE's network. KDHE's network configuration is state-of-the-art and used by many leading organizations, said officials.
KDHE reported that a new SAN was shipped from the vendor August 12 and arrived the next day, along with vendor personnel who began working with KDHE staff on installation.
Systems with the most public impact were to be given priority, said officials. As of Aug. 16 systems including Women, Infants and Children Program (WIC), Immunization Registry, Child Care Licensing, Kansas Information for Communities, Lab Certification and a few others, were reported as being up.
But with 150 servers and 25 terabytes of data, restoration is a time consuming and ongoing process, officials said.
Consequently, the Associated Press reported yesterday that about 120,000 birth and death certificates and other documents including immunization records had to be retrieved from where they were being stored in a central Kansas salt mine in order to give parents the information they required for sending their children back to school.
KDHE spokeswoman Kristi Pankratz told the AP the computer hardware storing all agency records that had failed had been fixed, but that it was still restoring its electronic data. Local news reports say a full system fix could run around $700,000.
"As a parent, I understand the need for documents to enroll your child in school and the frustration with not being able to attain them," said Bremby. "This has been hard for everyone and we appreciate your patience."