Mixed Reality in Orthopaedic Surgery

AAHKS Digital Health Committee
Digital Health White Papers

Mixed reality refers to the combining or blending of real‐world objects, people and scenarios with the virtual world. It is a continuum from the real world, where we all currently reside and interact, with virtual reality in which everything is completely virtual. Mixed Reality occurs when real‐world environments, physical laws, or multiple participants are involved in a virtual scenario.[1] There are several ways that people have used the term “mixed reality” throughout the years, although more recently it has been considered a part of the broader terms “immersive technology” or “extended Reality” (xR). These are described as, “technology that allows for the blending to varying degrees of simulated imagery and sound into one’s environment with similarly varying levels of interactivity and mobility.”[2]

First developed for real‐world use by the U.S. Air Force in the 1990’s, mixed reality can be used for training and simulation of real‐world tasks set in a virtual scenario with infinite possible variables. Studies have shown that human performance can be significantly improved when training involves virtual tasks overlaid on real environments.[3] Since then, this technology has made its way into multiple arenas including interactive product development, military training, healthcare training, remote working, functional mockup creation and continued use in aviation.

Use in Health Care

The most obvious benefits of this technology in health care are in enhanced training. Whether this is provider training where medical simulations can be overlaid in real hospital environments, or the myriad of opportunities for the enhancement of surgical training, mixed reality is likely to be one of the most important factors in the future training of health care providers. Immersive technology use has already shown promise in improving surgical training, demonstrated by virtual reality surgical platform systems, which gather data on the improvement in performing real world surgical tasks after training in a mixed reality environment (Image 1). [4]

Mixed reality technology also has the potential to allow surgeons separated by large physical distances to work together in training or learning capacities using a mixed real and virtual environment. There are limitless long‐term possibilities for the application of this technology in the provision of health care, especially if combined with artificial intelligence; however, for now they remain untested.

1‐ P. Milgram and A. F. Kishino (1994). “Taxonomy of Mixed Reality Visual Displays”. IEICE Transactions on Information
and Systems. pp. 1321–1329.
2‐ Justin Barad, MD (CEO, Osso VR)
3‐ Rosenberg, Louis B. (1992). “The Use of Virtual Fixtures As Perceptual Overlays to Enhance Operator Performance in Remote Environments”. Technical Report AL‐TR‐0089, USAF Armstrong Laboratory, Wright‐Patterson AFB OH, 1992.
4‐ http://www.telemedmag.com/article/need‐know‐virtual‐reality‐healthcare/