Healthcare can learn a lot from the finance and retail sectors, but there is another industry that hasn't been mentioned where healthcare, particularly electronic health record systems, can take some notes: the gaming industry.
A large part of the reason the gaming industry has captivated an entire generation, including my son, is their extensive user testing. Several years ago, my son was interested in buying the game Halo 3. Since the game is assigned an 'M' for mature rating, I had to do my own research to see if it was appropriate for him.
While I was a gamer in my college years, eight years of intense medical training and beginning my career had put me a little out of touch with the gaming landscape. The days of Atari and Nintendo were pretty simple. I soon realized that today's gameplay had transformed from a D-pad and two buttons and pixelated graphics to complex controllers and cinematic storylines, and popular titles boast intuitive, addictive qualities.
One of the articles I read, in WIRED magazine, centered on Microsoft's user testing of Halo 3, which is unlike anything I've ever seen in the healthcare world. Microsoft hired a doctor of experimental psychology to lead its Halo 3 user testing in its labs, which are outfitted with a one-way mirror, video cameras, and wired controllers, so that every moment – action and reaction – is digitally recorded.
Microsoft analyzed more than 3,000 hours of play by over 600 gamers. Through weekly tests and user heat maps, they found snags that stopped users in their tracks, like a "mutant alien that was far too powerful" or "a lava pit that too many players fell into." They also paused users frequently with pop-up one-question surveys to see how engaged, interested, or frustrated they were at that moment.
WIRED described this usability testing as "an awfully clinical approach," which is when it hit me: I was struggling with clunky EHR software in my local hospital to save real, human lives during the day, yet I could effortlessly interact with complex gameplay options to take alien lives during the evening? Why aren't EHR vendors and hospitals alike doing more to prevent physicians from falling into EHR lava pits? How can EHR vendors know when a doctor feels frustrated every time she hits a particular screen?
Perhaps even more importantly, how are individual hospitals and clinics measuring how well their users are adopting the local tweaks and enhancements made to the EHR vendor's initial configurations? In my 15 years of working with EHRs, I have yet to see any hospital or health system settle on the EHR vendor's out of the box configurations. So, even if an EHR vendor does engage in usability studies, there are still significant changes that occur at the local level that will affect "gameplay."
And the reality is that most EHR vendors aren't even participating in thorough user testing at the vendor level. According to a 2015 JAMA study, EHR vendors are falling short even on the user testing required by federal design rules. The researchers examined usability test results from the 50 most commonly used EHRs.
An incredible 18 percent did not have a public report of usability testing on file with the Office of the National Coordinator for Health Information Technology. Of the 41 vendors who did file, about one-third did not state the type of user-centered design process. Most alarmingly, a mind-blowing 63 percent engaged fewer than 15 participants in end-user testing. Further, 17 percent didn't have any physician participants in the usability tests of their computerized physician order entry systems.
ONC has stated that user-centered design processes and testing must be applied to a dozen primary EHR capabilities, but even if vendors comply, this is often done at the very end of the development cycle. EHR vendors need to do more to involve clinicians in the beginning phases of developing new features, so they are truly designed with their workflows in mind. It has taken years, but a few are starting to immerse staff in customer facilities and follow and track clinicians as they use their EHR system.
One well-known EHR vendor now has a staff of about 400 focused on product usability and are taking advantage of clinician feedback during breakout sessions at large industry conferences. They're also beginning to test eye-tracking software and other user-testing technology.
On the provider side, hospitals and health systems also need to make sure implementation and training times on new systems aren't rushed, so clinicians have the time they need to adapt to imperfect technology. One physician and workflow designer once told me that "I can't learn everything in one sitting, but the EHR's design should be 'figure-outtable.'"
Until the EHR design is a little more intuitive, providers need to build in those many "sittings" for training and be prepared to deal with clinician frustration and polarization. As one of my favorite reference books, The Inmates Are Running the Asylum: Why High Tech Products Drive Us Crazy and How to Restore the Sanity, notes:
"High cognitive friction polarizes people into two groups. It either makes them feel frustrated and stupid for failing, or giddy with power at overcoming the extreme difficulty. These powerful emotions force people into being either an 'apologist' or a 'survivor.' They either adopt cognitive friction as a lifestyle, or they go underground and accept it as a necessary evil. The polarization is growing acute."
Providers cannot place the sole blame on the EHR vendor for poor usability; they must do their part to encourage adoption and avoid polarization. A lack of thorough user-testing for EHR systems not only frustrates physicians and nurses on a daily basis, but it also is detrimental to patient safety and in delivering the quality care our patients deserve.
Microsoft's user testing was able to reveal flaws and bottlenecks that could be fixed in time for Halo 3's release. We can't put the genie back in the bottle, but we can start making positive steps, on both the EHR vendor and provider sides, to make future versions more user-friendly and take EHR systems to the next level. If user testing is taken that seriously for play, healthcare should engage in diligent user testing for systems that help save lives.