The Metaverse should be a place for impossible, dangerous, expensive, and even counterproductive activities, according to a VR expert who has spent over 20 years dividing his time between real and virtual worlds.
Meta and other companies betting big on the Metaverse hope users will use VR headsets for just about everything, says Jeremy Bailenson, Founding Director, Virtual Human Interaction Lab, Stanford University. Meanwhile, media and industry analysts have quickly pointed out the apparent lack of interest in Meta’s virtual reality offering.
“Both are missing the crucial point,” says Bailenson. “VR shouldn’t be used all day long, or even every day. Its strength has always been in its ability to provide us with special experiences, not with unending engagement.
“I’ve been working with the medium of virtual reality for more than two decades, and we have a simple saying in the lab, one we use after particularly frustrating days of wrangling hardware and software: VR is hard.”
Swapping the real world with a virtual one involves replacing light to the retina and sound waves to the ears. This has to occur a hundred times per second and be highly responsive to body movements. And like the physical world, virtual reality objects also need to adjust in size and volume when the user leans closer, and even the slightest micromovements can affect perception. The whole thing can be exhausting.
“When I first started working with virtual reality in the late 1990s, a system cost hundreds of thousands of dollars, and required a dedicated room with a full-time engineer to operate,” says Bailenson. “Headsets were clunky, heavy, and slow, and we had to keep a bucket in the lab for extreme cases of simulator sickness.
“These days, there are millions of standalone devices sold each year, which bundle the tracking cameras, render computer and optics display all within the headset itself, cost a few hundred dollars, and weigh less than a kilogramme.”
VR takes a toll on the perceptual system
Despite the more comfortable hardware compared to its early days, prolonged use of virtual reality can take a toll on the perceptual system. The difference between VR displays and the real world can cause fatigue over time. To address this issue, a 30-minute rule is enforced in some labs, which requires users to take a break, drink water, and remove the headset after half an hour of use.
And some individuals cannot handle VR even in short durations, particularly those who experience motion sickness in the passenger seat of cars. Being completely detached from the real world also poses a significant challenge. Being unable to see or hear one's surroundings for extended periods is not ideal for one's well-being.
“Ignore the hype, which says we’ll all live in the metaverse,” says Bailenson. “Actually, let’s use VR hardly at all.”
Bailenson’s lab has developed a framework, referred to as DICE, which concludes that VR is most useful for experiences that would be "impossible," "dangerous," "expensive," or "counterproductive" in the real world.
For example, training firefighters, rehabilitating stroke victims, studying art history through sculpture museums, and exploring time travel for a better understanding of climate change are all experiences that align with the DICE framework. Conversely, tasks such as checking emails, watching movies, or general office work are unsuitable for VR.
“During the pandemic, I was able to put DICE to the test at Stanford,” says Bailenson. “The university bought 200 Oculus Quest 2s for my Virtual People class, which I’ve taught since 2003. In four separate courses over a two-year period, more than 500 students learned about VR in VR, across various platforms.
“We logged hundreds of thousands of minutes in-headset together. It was an incredible endeavour, and exciting to finally get to use VR at scale. We spent more than a year planning the course, and quickly realised our biggest challenge wasn’t hardware or software.”
It was figuring out what to do during all those hours in the headset.
In the history of VR research and coursework, no one had ever put hundreds of people in VR for extended periods like this. During the course, the lab found that the DICE framework effectively kept participants engaged for long hours. To fill the many hours, the lab incorporated activities such as floating on yoga mats into space, learning paced breathing from a meditation expert while gazing at Earth, and building entire cities block by block in an empty virtual world.
“We experienced life in other bodies, practising empathy while wearing avatars of different skin colours or genders,” says Bailenson. “We experienced a corner kick from the goalie’s perspective in a German national football team training session to learn body position and posture. We time-travelled by visiting past recordings of our own avatars and walking around like ghosts inside a rendering of our recent social past.”
In contrast, activities such as "knowledge work," which mainly involved talking in meetings, working on shared documents, and watching videos, were ineffective in VR. The lab found that these activities had very little value in the virtual world, as laptops could perform them just as well, if not better, while VR headsets were more of a hindrance.
“One of the best use cases I have ever seen in VR is Penumbra’s REAL System, a full-body tracking system used by thousands of patients in hospitals and clinics across the country for rehabilitation for stroke, orthopaedic injuries and other conditions,” says Bailenson.
VR allows virtual simulations to dictate movements' exact speed and location to choreograph incredibly detailed trajectories. “And, of course, the immersive nature of the sessions maximises engagement and pain distraction,” says Bailenson. “Those patients are reaping huge benefits from just a few short sessions each week. In this regard, a little virtual reality can go a long way.”