As 2017 wound down, Amazon’s Alexa voice assistant rose up to the top of Apple’s App Store while Amazon’s Echo Dot was its own bestseller.

But some of the millions of those devices sold have already found their way into healthcare organizations like Beth Israel Deaconess Medical Center in Boston, Northwell Health in New York, the Commonwealth Care Alliance in Boston, and Libertana Home Health in Los Angeles.

“When we went from laptops to smartphones as our primary means of doing computing, that was a major paradigm shift,” said John Halamka, MD, CIO at Beth Israel. “Ambient listening tools probably will replace mobile devices.”

Amazon is hardly alone in the market. Apple Siri, Google Home and Assistant, and Microsoft Cortana are also available. Among hospitals undertaking pilot projects, Amazon reigns supreme — for now. At least one hospital is already girding to implement Google Home as well.

Early work at BIDMC

Halamka’s team is conducting extensive early work with Amazon’s Alexa voice assistant.

Halamka’s team has built a variety of skills for the Alexa voice assistant. A skill is the computing manifestation of a task that a voice assistant performs in the real world. Halamka said the Alexa APIs used to build skills are straightforward and easy to use. Beth Israel Deaconess System now has a variety of skills used in the voice assistants in patient rooms.

“If you are an inpatient, what are the things typically you would like to know?” Halamka said. “When will my doctor be here? What’s for lunch? Simple types of things for which you need answers. Today, you pull a cord installed in 1955 that flips a relay that turns on a light that means a nurse might be there in 10 minutes. Even modern nurse call systems still are very reminiscent of things that would have been in hospital rooms in the 1950s.”

Halamka questioned why healthcare professionals still think in that particular workflow when there is data in electronic form to answer these questions.

Now, patients can say, “Alexa? Ask BIDMC to call a nurse.” And Alexa will respond, “OK, I have just sent a request for a nurse.” Or, “Alexa? Ask BIDMC what is my diet?” And Alexa replies, “You are restricted to a bland diet for the day. To order a meal, call extension 12345.”

“Why should it be any different than that?” Halamka asked. “You develop these use-cases for which questions a patient typically asks, ones that can be asked through micro-services plumbed through the Alexa API.”

Northwell’s Alexa skill: Determine ER wait times

Northwell Health currently is using Alexa but soon will add Google Home into the mix. The provider organization is experimenting with many different use-cases for voice assistants. The big one currently is an Alexa skill that helps users identify the wait times at emergency rooms and urgent care centers that are near a given Zip code.

“You can have Alexa ask Northwell for the shortest wait time near your Zip code or check the time at a specific location,” said Emily Kagan Trenchard, vice president of digital and innovation strategy at Northwell Health. “The skill will give you the wait time as well as the address for that location and direct users to an alternative if the shortest wait time is elsewhere.”

It’s also smart enough to handle disambiguation around places with similar names. For example, Northwell has three hospitals known as Long Island Jewish – Long Island Jewish Valley Stream, Long Island Jewish Forest Hills and Long Island Jewish Medical Center.

“Imagine a scenario where you are in the kitchen and you cut your finger while cooking,” Trenchard said. “In this instance, you know you need to get medical attention and possibly a few stitches. You would say, ‘Alexa, ask Northwell what’s the shortest wait time near 11021?’ Alexa would then query the database of our emergency and urgent care wait times, which are refreshed every 15 minutes, and look at the locations nearest to the Zip code 11021.”

Of those locations, it would then calculate the shortest wait time and report back the location’s name, the wait time and the address. Say the result that comes back is for a Northwell GoHealth Urgent Care location, but the patient would prefer to go to the emergency department at their preferred hospital. In this case, the patient could say, “Alexa, what’s the wait time at North Shore University Hospital?” In response, Alexa would provide the wait time for that specific location.

Commonwealth: Patients drive voice tech growth

Commonwealth Care Alliance is using Alexa technology, but the organization’s plan is to ultimately access all available voice assistants to create individualized care environments for its members and patients.

“We believe this area of patient engagement will continue to grow – driven by both patient experience and feedback,” said John Loughnane, MD, chief of innovation at Commonwealth Care Alliance, which uses voice technology systems from Orbita. “We also see the great potential in this approach and technology to enable our members with physical disabilities to have more control over their environment.”

Commonwealth Care Alliance currently is leveraging Alexa to enable its members to perform a variety of tasks. Members can conduct hands-free calling to other people who have Alexa. They can conduct “drop-in” visits with members who agree to give each other access. This is important for individuals who need to check up on someone from another part of a home as Alexa acts as a built-in intercom system.

Members also can do an initial set-up of Google Calendar for personal care attendant schedules. The technology can then check on schedules to see who is going to be on the next shift. They can set reminders for routine tasks such as taking medications, setting up doctor appointments and other medical events.

A Commonwealth Care Alliance member would commonly use voice assistant technology to set up a calendar or schedule for their personal care attendant. A typical exchange might take place as follows:

Patient: Alexa, create an event: PCA #1 at 9 a.m. to 12 p.m. on Monday.
Alexa: Does this repeat?
Patient: Yes.

At this time, the personal care attendant shift is entered into the Google calendar. Once the Google calendar is complete, it is shared with the personal care attendant so they are aware of when their shifts are and who they may contact to arrange for coverage if shifts are missed.

The patient then can ask Alexa, “What are my next five events?” Alexa would inform the patient who is scheduled for their upcoming shifts. This is a powerful use of voice technology because the patients don’t have to keep asking others to check the usually written schedules.

Libertana: Voice tech helping independent living

Libertana Home Health is using Alexa along with the Amazon Echo Dot. Its goal is to empower people to live independently.

“Given our client population, the ability to use their voices to activate assistance and get reminders to stay on task makes the likelihood of aging in place a reality,” said Debra Harrison, RN, assisted living waiver public subsidized housing manager at Libertana Home Health. “The inevitable physical and mental deterioration of the aging process is a difficult adjustment. Having the right assistance in the comfort of their own homes is invaluable in helping our clients acclimate.”

The pilot study has proven that Alexa becomes a “friend” to the client, Harrison added. And very important, it helps break the loneliness that most experience.

“The client with shingles whose pain makes moving unthinkable can use her voice to play music, relax and stay comfortable,” she added. “The use of cognitive games makes the clients think and laugh, the ability to set a timer, ask for directions, and hear a response makes them feel less isolated.”

A patient can wake up in the morning and say, “Alexa, open Libertana.” The Libertana app is opened and there is a morning greeting individualized to that client. Alexa then continues with reminders such as taking medications at specified times and putting on their life alert pendant.

“The voice device shares information about activities happening in the facility throughout that day and provides reminders for them at various times to take specific measurements, blood pressure or blood sugar checks, for example,” Harrison said. “Reminders also are provided to exercise, drink fluids and eat snacks.”

The most widely used feature, which can help patients gain peace of mind, is the ability to request assistance from Libertana staff. With the voice assistant, a patient does not need his or her mobile phone nearby. After finally getting comfortable in a reclining chair, for instance, the patient might realize he or she left the mobile phone in another room. With Alexa, the patient need only speak to get the assistance needed – perhaps getting to the bathroom safely, asking someone to get pain medication or reporting feelings of lightheadedness.

“At the beginning of the pilot, which we limited to one of our facilities, we identified that some clients were unfamiliar with or forgot how to activate the skill,” Harrison explained. “We spent time early on to educate users on use of the device; essentially, reinforcing their need to say, ‘Alexa, open Libertana.’ Not all had this challenge, but of those that did, once they overcame it, all program participants have become very comfortable using Alexa.”

The HIPAA situation

None of the voice assistants yet have HIPAA business associate agreements to adhere to the HIPAA privacy and security rule. As the Office of Civil Rights will tell a healthcare organization, however, as long as there are none of the 18 identifiers passing over a clear channel, the organization is safe.

“Just telling Amazon here is a patient in 701 who wants to know when lunch arrives, there is no HIPAA information going over that kind of dialogue,” Halamka said.

Ultimately, if one can just be somewhere and ask a question, and the question is a skill on the back-end powered by micro-services that are attached to a patient’s record or to a dietary system, that is so much friendlier for all involved, Halamka said.

How-to tips for using voice assistants

Commonwealth Care Alliance, Libertana Home Health and Northwell Health are pioneering the use of ambient listening voice assistants in healthcare, and they all have so far achieved success with the technology as measured by patient satisfaction.

When asked to share some “how to” tips with their peers at other healthcare organizations looking to get into the voice assistant game, the executives at these three organizations were overflowing with help.

Emily Kagan Trenchard, vice president of digital and innovation strategy at New York health system Northwell Health, offers four tips to getting started with a voice assistant project.

1. Experiment away: “First, don’t be afraid to start experimenting because both Amazon and Google now have SDKs that are open to anyone to use – you can try 50 different skill ideas before you land on one that your end users might really love,” Trenchard said. “

2. Don't expect perfection: Working with voice assistants is an easy thing to start, but a harder thing to perfect. Plan on not only having developers on the project, but user experience and/or copy people to help generate the utterances. This will likely take up the bulk of your time once you determine the nature of the skill you want to build.”

3. Utilize your APIs: A healthcare organization does not need anything beyond its existing website to get started, she advised. Northwell has its Alexa skill running off of its content management system and taking advantage of the same APIs that the website and digital signs use. That said, there are more tools available to help visualize workflows, manage utterances and even cross-compile code for Google, Alexa and other voice assistants. Once a healthcare organization gets up and running, these tools could be a great help, she added.

4. Step-by-step: “And fourth, think about small, simple tasks and information requests,” Trenchard said. “If it’s going to take more than one or two steps to get the information a user is looking for, it might be better accomplished in a visual interface than in voice.”

John Loughnane, MD, chief of innovation at Commonwealth Care Alliance, which uses voice technology systems from Orbita, advises healthcare organizations trying to figure out how to get voice assistants off the ground is to make them valuable.

“It is important to keep in mind that voice assistants must support members and patients in their everyday lives and add value,” Loughnane said. “When looking to integrate new technologies and solutions such as voice assistants, it is critically important to collaborate with others across the care continuum in order to determine best practices for treating different populations.”

Through this collaboration, these approaches can be modified or adapted to work in different situations so everyone can benefit, he added.

Prime your users

Debra Harrison, RN, assisted living waiver public subsidized housing manager at Libertana Home Health, which uses voice technology systems from Orbita, says organizations looking how to make first-generation voice assistant projects work should choose their locations and users wisely. Do everything possible to have individuals who are open to using such technology, even if it means coaching.

“In my experience, that is the key,” she said. “It’s also important to knowing ahead of time that daily reinforcements are necessary for a window of time. Ensuring there is little to no pressure or stress involved with them using it is also important. We portrayed this as something that would be fun and helpful.”

For the first few times using it, patients were not actively involved – they did nothing but watch the healthcare professionals doing the coaching. But that helped alleviate the feeling of pressure patients might get from the fear of doing something wrong.

“They let us know when they were comfortable using their own voices when they saw how easy it was – and it took off from there,” Harrison said. “Giving the clients the time they need in the beginning phase, however long, will help them use it successfully and for the long run.”

The results are early

It remains very early days for the use of voice assistants in healthcare. Consequently, provider organizations experimenting with the burgeoning technology do not have many hard results like return on investment to report. But these organizations are showing what the technology can do, and have patients to back them up.

“We’ve proven the technology is very robust,” BIDMC’s Halamka said. “In all of our testing and piloting, the technology works. We’ve proven the development time to create these things is very short, because Amazon has really democratized the technology, you do not write programs, you write scripts and configuration files. It’s the kind of thing a person can do in a weekend.”

The use of the skill has been growing steadily, Northwell’s Trenchard added.

“So far we’ve seen over 1,000 utterances captured in hundreds of sessions,” she reported. “I think it’s still a very novel idea that consumers can find this kind of information from Alexa. From patients we’ve spoken to, they feel it gives them a sense of comfort to know how long they will have to wait when they walk in at an urgent care center or emergency room before they leave the house.”

Voice assistant technology also has been a great way for Northwell’s internal development and user experience teams to experiment with voice interfaces and learn from the feedback from real-world users.

“There’s no substitute for having live user experience data when exploring how a new technology could serve our patients,” Trenchard said.

Libertana Home Health does not yet have documented results. The pilot study’s purpose was threefold.

“To see if clients were able to use it consistently, benefitted from it, and felt empowered by using it,” said Harrison of Libertana Home Health. “We did see results in all areas. One client with a cultural based fear of using Alexa, due to having Big Brother watching, would not use it in the beginning. I actually thought he was a lost cause. A few months later, however, with constant support from our staff, he saw its benefits and now uses it on a daily basis.”

Commonwealth Care Alliance currently is collecting data and expects it to be available in the first quarter of 2018. In terms of anecdotal results, members have reported they find the use of voice assistants makes it much easier for them to call and create schedules, giving them more autonomy in their lives, said Loughnane of Commonwealth Care Alliance.

While provider organizations experimenting with ambient listening voice assistants may not have a lot of hard data results to report quite yet, they have already learned lessons to share with other providers considering voice assistants.

“We’ve learned that voice assistants need lots of training,” stressed Trenchard of Northwell Health. “We have over 1,500 utterances logged to date and there are still things that Alexa sometimes does not know how to process. We’ve also learned that Alexa is not very good at dealing with accents. We have high hopes for voice assistants in the future.”

Today, commercially available voice assistants are intended for group settings, and that makes security a very tricky thing to tackle, especially when dealing with sensitive health information. While newer releases have shown that voice agents can get better at recognizing which individual in a household might be speaking, the technology is not yet sophisticated enough that Northwell feels comfortable sharing care information with patients through the devices. For now, Northwell is focusing on use-cases that don’t involve sensitive data.

At Beth Israel Deaconess System, the early feedback from patients, board members, administrators and caregivers, is that the technology is “fabulous,” Halamka reported.

“The idea that instead of using a pull-cord or a phone or an e-mail, that you can just in the room have ambient listening and actions happen reduces the burden on everyone,” he said. “How much burden do people in healthcare feel right now in the era of post-meaningful use HIPAA Omnibus rules? Burden seems like the keyword of 2017. The idea that this is a real time-saver and burden-reducer is a real lesson learned.”

The impact on IT is very low – a healthcare organization does not need to find someone new or spend a lot of money, and that is an important lesson learned, Halamka added.

“The skills that are necessary to do this are really just scripting, not programming,” he explained. “The technology is mature, really low cost, low impact, easy to deploy. So, be aware of the privacy issues, given that it is early. Amazon will eventually get to HIPAA compliance on Alexa, they are just not there yet.”

The future: Patience with patients

Harrison at Libertana Home Health cites lessons learned that revolve around the patient.

“We realize that our clients need reinforcement in learning how to open the Libertana app,” Harrison said. “Initially, this has to be done on a daily basis. This was simple for us since we have caregivers going into apartments on a daily basis. We learned the need to be patient, and were pleased with the pace of adoption once we crossed the learning curve.”

Libertana also learned that it needed to choose patients carefully. For the pilot, success was in part likely due to the time spent reviewing clients’ personalities, their willingness to participate, their general compliance in their care, and their cognitive ability, Harrison said.

Beth Israel Deaconess, Northwell, Commonwealth Care Alliance and Libertana Home Health have proven that voice assistants can play a very helpful role in the delivery of healthcare. They also have shown there is some work yet to be done for these voice tools to proliferate within the four walls of healthcare organizations.

Ultimately, voice assistants hold great promise for healthcare — promise that may be realized sooner than many now think.

Twitter: @SiwickiHealthIT
Email the writer: bill.siwicki@himssmedia.com