Philips and Microsoft developed an augmented reality concept for image-guided minimally invasive surgeries, the companies said.

Based on Philips’ Azurion image-guided therapy platform and Microsoft’s HoloLens 2 holographic computing platform, the augmented reality applications include those for image-guided minimally invasive therapies.

WHY IT MATTERS

Built for HoloLens 2, the augmented reality concept brings live imaging and other sources of vital data — currently displayed on large external 2D screens — into a 3D holographic environment that can be controlled by the physician.

For example, one concept allows surgeons to see the real world superimposed with the live data and 3D medical imagery needed to guide precision therapy.

The platform lets doctors control Azurion with voice recognition, eye tracking and advanced gestures. The HoloLens itself is a self-contained holographic computer that enables hands-free, heads-up interaction with 3-dimensional digital objects.

The concept, which Philips said is being used to gather further clinical insights, is designed for the operating room of the future, and will help support the development of commercially available augmented reality solutions for use in image-guided procedures.

In such minimally invasive procedures, physicians cannot directly see and touch the treatment area, but instead rely on ultra-low dose X-ray imaging and other navigation technologies to see inside the patient and guide their actions.

ON THE RECORD

“By collaborating with Microsoft and HoloLens 2 we can take it to the next level, immersing the physician in a tailored augmented reality environment,” Atul Gupta, chief medical officer for Image Guided Therapy at Philips, said in a statement. “On our Azurion platform we seamlessly integrate a range of data sources in a way that’s intuitive to understand and control.”

The Azurion platform is powered by Philips’ proprietary ConnectOS, which combines technical innovations in both software and hardware-- ConnectOS allows the integration of advanced digital innovations on the platform.

“Mixed reality is giving people new ways to interact with the digital and physical world, bringing the benefits of the digital revolution to entirely new experiences across the globe,” Alex Kipman, technical fellow, AI and mixed reality at Microsoft, added. “Mixed reality holds great potential in healthcare, and our collaboration with Philips shows how that potential is already beginning to be realized.” 

Nathan Eddy is a healthcare and technology freelancer based in Berlin.

Email the writer: nathaneddy@gmail.com

Twitter: @dropdeaded209