Revolutionizing Healthcare? Apple’s Vision Pro and Its Impact on EHR Systems

“Revolutionizing” is a word that often gets overused but, in this case, it may be appropriate given the transformative capabilities of Apple Vision Pro. Undergirded by visionOS, Apple’s mixed reality operating system, Apple Vision Pro is phygital—seamlessly combining digital content with physical world realities to create a personalized, immersive user experience.

Developers therefore have “unparalleled opportunities” to innovatively and positively affect people’s lives in areas ranging from medical imaging to surgical planning, behavioral health, training and education, and more—including electronic health record (EHRs) systems.

Here’s more about the spatial computing it uses along with real-world use cases from top healthcare systems.

Elements of Spatial Computing

Spatial computing technology allows computers to naturally blend with the physical world, with users experiencing this “phygital” combination through special headsets such as those provided by Apple Vision Pro. These devices allow users to observe the real world while also embedding objects into the images in ways that look three-dimensional.

Three key technologies merge to provide this experience:

1. Computer Vision: This takes inputs from cameras and other sensors and processes the “position, orientation, and movement of objects” in the environment.
2. Sensor Fusion: Data from multiple sensor inputs are merged, creating a comprehensive, accurate environmental view.
3. Spatial Mapping: This creates a 3D model, allowing objects to be more precisely placed and digital content to be manipulated.

Spatial computing “understands” an environment’s depth, which allows the technology to place and manipulate virtual objects in ways that mesh with the real world. Humans can interact with virtual objects because the technology contains an advanced system to track eye movements; when the person looks at an object, they can manipulate it by simple movements such as a wrist flick or pinching of fingers. Speech recognition features allow users to also use voice commands to achieve goals.

Stanford Medicine: Enhancing Surgical Practices

On February 29th, 2024, Stanford Health Care announced how Apple Vision Pro is being tested in operating rooms with cardiac electrophysiologist Alexander Perino, MD conducting an ablation procedure that treats heart arrhythmias with the technology. Typically, Dr. Perino needs to look at multiple monitors displaying real-time data in operating rooms without enough space for them to be conveniently located. Information projected on them can be difficult to review, often requiring inefficient workarounds.

Using the Apple Vision Pro headset, Dr. Perino efficiently manipulated the virtual monitor containing all relevant information for the procedure. Both Stanford and Dr. Perino expressed delight about this pilot usage, with the organization considering how else to use Apple Vision Pro. Possibilities include more advanced operating room applications and for training and educational purposes.

Dr. Perino was quoted as saying that he hopes this establishes the technology as one that can “reduce barriers to quickly and easily review and manipulate intraprocedural data, increasing efficiency and clinician enjoyment. Ultimately, translating these improvements can help our patients.”

Cedars-Sinai: Behavioral Health Applications

Cedars-Sinai has launched a behavioral health app—eXtended-Reality Artificially Intelligent Ally (Xaia)—to provide patients with “self-administered, AI-enabled, conversational therapy in relaxing spatial environments.” This immersive therapy session can take place in a meadow by a creek or on a sunny beach while patients meditate or do deep breathing exercises. A trained digital avatar guides the patients.

Professor of Medicine, Director of Health Services Research at Cedars-Sinai, and Xaia co-founder Brennan Spiegel calls Apple Pro Vision an innovation that is a “quantum leap beyond previous technologies.” Psychiatrist and Xaia co-founder Omer Liran, MD calls this a “transformative step in making quality therapy accessible to all.”

As another crucial step forward, although Xaia is built upon advanced technology, this platform allows the technology to take a backseat during the patient experience so that the healing journey is front and center. Plus, Xaia is designed to “democratize mental wellness” that focuses on each individual’s unique experience.

Boston Children’s Hospital: Training and Education

Boston Children’s Hospital is leveraging the power of Apple Vision Pro’s technology in its CyranoHealth app that creates an immersive training experience with lifelike simulations. Nurses and other healthcare professionals can use this app to learn about and practice working with virtual medical equipment. By creating this “safe, universally accessible virtual learning environment,” the healthcare system hopes to help alleviate fatigue and burnout in their workforce while enhancing their skills and readiness.

Brennan Spiegel of Cedars-Sinai also offers insights into the benefits of this technology when providing training. Because spatial computing can facilitate emotional and psychological responses—perhaps when medical students or professionals experience a virtual code blue—this enhances learning possibilities in ways that other technologies can’t. To demonstrate, Spiegel suggests that people think about their strongest memories, noting how they’re likely tied to an emotion. People learn more effectively when they can imagine an experience in their heads, like they can with technologies derived from Apple Vision Pro.

Sharp HealthCare: Healthcare IT Integration

Integrating Apple Vision Pro’s technology into EHR systems comes with vast potential for improved patient outcomes. So, Sharp HealthCare collaborated with Epic Systems to investigate how Apple Vision Pro can be used in healthcare applications. This has resulted in the introduction of the Epic Spatial Computing Concept that facilitates the ability of physicians and clinicians to smoothly, efficiently, and intuitively chart, review labs, securely chat, and address In-Basket workflows. This can be accomplished through eye tracking and gestures as well as through voice recognition.

Dr. Brian Lichtenstein, who helped to launch the Spatial Computing Center of Excellence at Sharp HealthCare, calls Apple Vision Pro a “phenomenal tool for people who need to touch anatomy in some form.” He expects this technology to become a widely used presurgical tool so that surgeons can proactively understand a patient’s unique anatomy and be prepared to address those nuances in the operating room.

Future Implications for Healthcare

Potential advancements and expansions of Apple Vision Pro are numerous, including the following:

1. Telemedicine: Patients would receive a more interactive experience than with today’s 2D teleconsultations. Then, when the imaging potential of Apple Vision Pro is merged with artificial intelligence algorithms, remote diagnostics and patient monitoring can lead to more rapid medical responses in ways that reduce the burden on healthcare organizations.
2. Mental Health: Apple is testing its technology’s ability to monitor facial expressions, which could help to identify a user’s state of mind and conditions such as stress, sadness, anxiety, and PTSD. Researchers have already used virtual reality to diagnose and treat phobias and PTSD, and Apple Vision Pro could further their abilities to do so.
3. Surgical Procedures: Advancements can occur because of the real-time information gleaned through the tool’s tracking capabilities and the structures that surgeons can superimpose during procedures. Plus, professionals can communicate efficiently even when in different locations.
4. Physical Rehabilitation: This can take place through the immersive environments available to patients and the movement tracking and real-time feedback.
5. Medical Training: Students can explore human anatomy, examine patients, and practice surgical procedures in safe, controlled environments. They can also learn collaboratively even when participants are in distant locations.

As far as the future impact on EHR systems, a research study was conducted by experts in the University of Minnesota’s department of computer science and engineering in tandem with the Mayo Clinic’s department of oncology: Spatial Computing Opportunities in Biomedical Decision Support.

One key finding is that, when spatial computing is integrated into EHR systems, this has the
“potential to transform EHRs into dynamic, interactive maps of human health.” Healthcare providers would not only have information about the patient’s current health status but also have predictive insights, thanks to the spatial and temporal patterns of information. The result: healthcare professionals will be able to seamlessly navigate EHR systems to provide the best patient care.

When referring to the Epic Spatial Computing Concept, Dr. Lichtenstein shares how, today, information can be pulled into a space of virtual reality, thanks to technological advances. In the future, though, he believes we may be able to “use augmented reality to superimpose data in the real world.”

He applauds the incredible opportunities available in the spaces of anatomy, education, and preclinical care, adding that the industry is just at the “beginning of what will be a longer story about how we’re using both augmented and virtual reality to deliver care to patients.”

Transformative Potential of Apple Vision Pro in Healthcare

Collaboration is at the heart of these technological advancements as numerous professionals contribute to the development of Apple Vision Pro; the creation of healthcare apps that use Apple’s technology as their foundation; and the testing of these innovative applications.

As these technologies become increasingly integrated into EHR systems, the need for EHR consultants to effectively implement and integrate these applications is growing. This integration is crucial for enhancing patient outcomes and streamlining healthcare delivery. As a leading EHR consulting company, HealthTECH Resources is committed to being at the forefront, working alongside healthcare systems that embrace cutting-edge technologies.

Our expert team is always ready to discuss your unique healthcare IT needs and provide solutions to enhance your system’s efficiency and patient care. To partner with a leader in EHR consulting, contact us online or call us at (602) 903-7961 today to see how we can support your organization’s growth and success!