Mixed Reality in Healthcare: Is It Ready for Prime Time?
By Rishindra M. Reddy, MD, MBA and Jenny Bui, MD, MPH, Medical Innovations in eXtended Reality and the University of Michigan, Section of Thoracic Surgery, Ann Arbor, MI
Technological advancements in virtual (VR), augmented (AR), and mixed reality (MR) hold significant promise to revolutionize multiple industries, including healthcare. Currently, these developments are primarily utilized in the gaming and entertainment industries. Still, their applications in healthcare are vast, encompassing areas such as resident and student education, patient therapeutics, and telehealth. However, integrating these technologies into the US medical system poses significant challenges, including hardware and software familiarity, high costs, and integration into existing hospital IT systems.
Education and simulation are likely the lowest barriers to entry for adopting this new technology. MR and VR training paradigms have already been developed for procedural training (i.e., intravenous line placement, bladder catheter placement), surgical training, and interactive virtual patient scenarios. Each platform offers unique benefits and challenges. MR programs that incorporate physical simulation equipment enable learners to engage in hands-on training with tactile feedback. It is critical for early-stage learners to develop technical skills and become familiar with the tools used in daily patient care. However, this approach requires significant resources, including equipment costs, coordination of materials, and designated physical space for training. In contrast, VR programs allow an immersive training environment that can be used anywhere. Despite this flexibility, current VR systems lack the capacity to deliver hands-on experience, which is critical for novice learners. As a result, VR scenarios are currently better suited for cognitive training (i.e., virtual patient scenarios) or for reinforcing technical steps among experienced learners (i.e., demonstrating surgical techniques to skilled surgeons who need to refresh their knowledge of specific procedures). To support broader adoption, focused research is needed to evaluate the effectiveness of these training paradigms, demonstrating that they can provide comparable or superior outcomes to traditional educational methods.
There are many exciting developments in the field of MR with a great deal of promise for patients and the healthcare industry.
These training scenarios, along with the companies that develop them, can avoid some of the more stringent hospital IT regulations since no patient data is involved. The Microsoft Hololens2 ©, a widely used MR headset across multiple industries, has been discontinued, with unclear plans for future development. There have been limited opportunities for laypersons to use this headset due to its high costs and limited applicability to the general public. In contrast, VR systems such as Meta’s Quest 2 and Quest 3 © have gained broader familiarity among the general population due to their lower costs and widespread applications in gaming and entertainment. Regardless of the hardware system, content creation remains a barrier. Developing mixed or virtual reality training scenarios requires specialized software, necessitating collaboration with software companies to produce high-quality, interactive content tailored for training purposes.
The use of eXtended reality, specifically VR, for patient therapeutic applications has seen exponential growth. The VA Healthcare system has been supporting several start-ups that use VR for post-traumatic stress disorder treatments and rehabilitation applications. While efforts are underway to show the clinical benefits of these treatment strategies, widespread adoption is hindered by similar challenges seen in educational training, such as hardware costs, IT approval, and both patient and provider familiarity with the equipment. There are active efforts to address these structural barriers. These include navigating the variable legal requirements across different healthcare systems and facilitating the integration of software and hardware into hospital IT infrastructure with support from technology teams.
Telehealth and virtual care are evolving with the integration of AR, VR, and AI interactive “chatbots.” These technologies aim to provide patients with a “virtual” provider capable of obtaining detailed histories, thereby saving time for the actual healthcare providers. There is also an opportunity for patients and providers to meet in a virtual space, limiting the need for patients (or providers) to travel to the hospital. Since COVID, there has been a robust transition towards virtual visits through video and phone calls. While VR is the next logical step for virtual care, its adoption faces barriers, including hardware costs for both patients and providers, as well as the technological support required for implementation. In my current practice, approximately one-third of patients struggle to participate in video visits due to internet limitations and a lack of familiarity with video applications. These challenges are compounded by the limited availability of technical support for patients, and introducing more complex VR-based technology will only risk exacerbating these issues.
There are many exciting developments in the field of MR with a great deal of promise for patients and the healthcare industry. However, several challenges remain, including a lack of widespread familiarity with the equipment, costs, and maintenance required and the difficulty of navigating technological and legal approval processes within the healthcare system. As hardware costs continue to decline and structural barriers within hospital settings improve, the integration of these technologies will likely become more feasible. Nevertheless, further research is needed to demonstrate the clinical and financial benefits of using AR and VR in these different arenas.