Virtual Reality Today and Tomorrow: Use Cases in Medical Treatment

Today's virtual reality (VR) is not just for gaming and entertainment venues. This technology is also increasingly being used in business.
At Fujitsu Forum 2018 on May 17-18, 2018, we held a conference entitled "Fujitsu's Virtual Reality Presents New Possibilities for Business: Virtual Reality Extends Beyond Reality's Boundaries Science Fiction Technology Made Real."
For this, we invited a science fiction writer and VR application developers from the healthcare field (Heart Simulator); they presented some key points about adopting VR in business and VR's future possibilities.

[Fujitsu Forum 2018 Conference Report]

How Will VR Change Our Lives?

VR Estimated to Become a 13 Trillion Yen Market by 2025

First, Fujitsu Chief Evangelist Iwao Nakayama explained the VR market.

Iwao Nakayama
Chief Evangelist, Global Marketing Group Corporate Executive Officer
Fujitsu Limited

Virtual reality is defined as "technology that uses science and engineering to create something non-existent by stimulating the user's five senses." At present, various manufacturers sell VR hardware, such as headsets like Sony's PlayStation VR, HTC's Vive, and Samsung's Gear VR. VR applications are also growing in terms of types and numbers.
VR is forecasted to grow rapidly from 2018 to become a three trillion yen market (27 billion USD), five times or more the present level, by 2022. Some predict that by 2025 VR will surpass the 12 trillion yen TV market to reach the 13 trillion yen level (115 billion USD).

Allow me to introduce some of the latest VR case examples. The ShelfZone service enables users to shop from home in VR stores, providing a shopping experience with impressive realism without leaving your own room. I think the world depicted in the video below will become commonplace.

ShelfZone VR Shopping Video

Facebook acquired Oculus in 2014 and released a social VR application, Facebook Spaces, in 2017. Facebook Spaces enables users to socialize with real-world friends using avatars in a VR world. In addition to these examples, many very attractive, accessible VR services have already been launched.

Cutting-edge VR/Mixed Reality (MR) Use Cases in Medical Treatment

Next, Dr. Maki Sugimoto, a leading authority on VR use in healthcare, took the stage and spoke about the latest VR/MR use cases in the field of medical treatment.

Maki Sugimoto, M.D./Ph.D.
Founder, Managing Director, and COO, Holoeyes, Inc.; Founder and CEO, Mediaccel, Inc. Visiting Researcher at the Research Center for Advanced Science and Technology, Department of Information Physics and Computing, The University of Tokyo

As technology to make medical doctors' work more efficient, I am paying close attention to VR, augmented reality (AR), and MR. Recently, the term extended reality (XR) is sometimes used to encompass all three.

We are unable to perceive 3D space with conventional flat 2D monitors. For example, in the field of medical treatment, when surgeons use a flat 2D monitor to look at images of the inside of the abdomen during endoscopic surgery, they cannot perceive the actual depth and positions of the organs accurately.
Because of this, some statistics show that endoscopic surgeries cause more organ damage than open abdominal surgeries.

To solve such problems, we have developed technology to see things "in space." One example is OsiriX, which reconstructs two-dimensional CT and MRI images to produce three-dimensional images.
The software uses data pieces called polygons to transcribe the shapes of individual organs and express them via coordinates, thereby enabling users to grasp cancer and veins intuitively in space.
Making use of the binocular parallax of the human eyes, OsiriX synchronizes different data for each eye to create a stereoscopic view. By donning a head-mounted display, the user experiences depth and an improved feeling of immersion.
We apply this VR technology in surgery. For example, surgeons can conduct pre-op practice and simulation using actual patient data.

Another example is the use of VR in Heart Simulator (*), which we co-developed with Fujitsu. Heart Simulator simulates the heart's complex behavior and disease conditions as well as the heart's shape. It provides a stereoscopic view of heart movement, blood flow, pressure, and electrical signal flow in VR.
Since this technology enables us to visualize how a certain operation will change blood flow and pressure, we may be able to use it to pass on veteran heart surgeons' know-how and techniques to younger doctors.

VR/MR Provides More Realistic Healthcare Education (Japanese)
  • *: We ran a trial to display the output data of a simulation in VR.

An example of more advanced technology is mixed reality (MR).
We are promoting use of MR in surgery as well.

Microsoft's HoloLens can constantly scan the environment around the wearer. So, if both an experienced surgeon and a younger colleague wear a HoloLens during surgery, they can share in real time information like "We will make an incision here" or "Be careful of the artery there." This has realized benefits such as shorter operation times and reduced patient burdens.
These technologies can also be applied to healthy people. According to one survey, 60% of expectant mothers worry about their pregnancies, which sometimes leads to depression and other symptoms. New research suggests that using XR to view an unborn baby makes expectant mothers grow attached to their babies and creates a bond between the mother and child before birth, thus preventing depression and attachment disorder.

Will Technological Advancement Realize the Future Portrayed in Sci-Fi Novels?

Following Dr. Sugimoto, science fiction writer Mr. Taiyo Fujii predicted the future of VR from a sci-fi perspective.

Taiyo Fujii
Science Fiction Writer

First please allow me to introduce myself. I used to develop 3D software, but one day I felt the sudden urge to write a novel. I started writing on a train using an iPhone and debuted as a novelist in 2012.

My first novel, Gene Mapper, is set in 2038 and concerns genetically modified crops. In the book, I devoted many pages to describing future person-to-person communication using VR.
The characters in the story wear special contact lenses to always be in a state of MR/XR. Many businesspeople use the contact lenses. For example, at a business conference, the contact lenses project the avatars of other conference participants in front of them, and the avatars talk with each other. That is the future I wrote about.

I put extensive detail into the novel. For example, delays of about 0.2 seconds to 1 second are inevitable in Internet communications such as Skype today. That is why you cannot sing in chorus while teleconferencing. In Gene Mapper, I write about a world where avatars correct this latency to facilitate smooth communication, remove glitches in conversation (such as ums and ahs), and automatically delete any remarks that may violate non-disclosure agreements (NDA) during a business conference.

I put a lot of effort into describing this technology to make such a future believable to my readers. In doing so, I found Fujitsu's body area network (BAN) webpage very informative (http://pr.fujitsu.com/jp/news/2014/02/12.html).
BAN is a technology that uses the human body as a network transmission path. I wrote the story imagining that BAN might be able to send megabyte-sized images to the contact lenses.

Like the future I depict in my novel, technology continues to change the world we live in. As we all know, since the iPhone came out in 2012, our communication styles have completely changed. I think VR will transform our lives still further.