Creating near-real experiences with IoT in the metaverse

Set to transform many aspects of human experience, the metaverse remains an ‘unreal’ concept for many. Harnessing disruptive technologies, the metaverse has the power to expand our capabilities, strengthen our connections and enhance our interactions with each other, bridging the gap between the physical and digital to offer a truly immersive experience.

However, one challenge that many technologists are facing is the new infrastructure needed to host such a complex range of technologies ‒ currently, such infrastructure isn’t yet ready to bring the metaverse to life.

For the metaverse to work, internet of things (IoT) devices need to be employed by the platform to collect data. To do this, the metaverse will connect hardware, controllers and physical objects ‒ all with the help of IoT.

“We’re at a really exciting and unique point in the evolution of the digital world where real feeling – seeing, hearing and feeling an experience – is becoming possible in the metaverse through synthetic data captured by IoT and connected devices. It’s through this ecosystem of connected devices that we can now render ‘real feel’ in the virtual space by creating digital twins and avatars that are contextualised in real-time,” comments Gopikrishnan Konnanath, Global Head of Engineering and Blockchain Services, Infosys.

As an example of this, we can take a look at devices worn on humans: geolocation tags have the ability to detect body movement, eye gaze and different body functions crucial to replicating a near-real persona, as well as creating a near-real and immersive experience.

Considering IoT technology is used to provide a physical interface to connect computers, allowing users to remotely interact with the world, it’s perfectly placed to imbue verisimilitude across the metaverse.

Jamie Boote, Software Security Consultant at Synopsys Software Integrity Group outlines: “The metaverse could move such interfaces from professional users to end-users. While not every end-user is going to want to interface with factory valves, the IoT is also pivotal in managing commuter lines and transportation for central conductors. A user interface enabled by the metaverse could take information collected by industrial IoT devices, tracking train locations and providing dynamic travel guidance as users move about their commutes.”

Combining IoT with AI and XR in the metaverse

Playing a pivotal role in data acquisition from physical assets, IoT can augment and amplify the immersive experience, especially if the technology is complemented with artificial intelligence (AI) and extended reality (XR).

“The convergence of AI, AR/VR, IoT and real-world data means the metaverse can be a strong enabler of what businesses are already trying to achieve – better customer outcomes and experiences,” says Harvey Lewis, Associate Partner, Client Technology and Innovation at EY.

“From a manufacturing perspective, IoT sensors enable us to create real-life mock-ups of factories, machinery, and products down to a microscopic level, which we can explore in intimate detail in the metaverse. This allows us to locate faults or make machines more efficient without having to exhaust any resources,” he continues.

As it bridges the gap between virtual and physical worlds, the metaverse will rely on IoT and the data it generates to provide an unrivalled feeling of immersion into the digital world, as Konnanath notes: “The data collected through sensors such as, cameras and LIDARs are critical for immersive and contextual experiences and bring intelligence to the environment so it can adapt to the needs of the users interacting in the digital space.” 

Via the metaverse, IoT technology will also enable more natural interactions in the workplace, using technologies such as facial and motion tracking for avatars to replicate, giving them a more lifelike characterisation.

“By providing an environment where we can have full-body representation instead of boxes on a screen (so we can finally see how tall someone is virtually), the reality of the metaverse gives us the impression of connecting physically, as well as virtually, with our colleagues,” says Lewis.

Digital twins creating near-real experiences

Not only will the combination of IoT, XR and IoT augment and amplify the immersive experience, it will also allow technologists to create digital replicas

With AI and XR, IoT has the ability to provide innovative use cases and highly immersive experiences through digital avatars and digital twins. Without this technology, the metaverse would not be able to render immersive experiences and the near-real feeling. 

Taking building optimisation through the metaverse, this technology makes it possible to better monitor existing building operations, such as heating, cooling and lighting, helping reduce global emissions.

“Digital replicas therefore play a big role in not only improving the efficiency of current processes, but also in driving sustainability and helping us achieve our climate goals. On the flip side, the fully-immersive environment of the metaverse can bring pertinent issues and challenges to life, such as climate change, in a way that makes it difficult for us to ignore,” notes Lewis.

Digital twins are fundamental to the success of the metaverse, empowering the user interface to better provide information that’s needed in a format optimised for the virtual reality or augmented reality environment they’re in. 

Particularly beneficial for the manufacturing industry, Konnanath explains how digital twins provide real value and benefit to the industrial world: “A key example of how digital twin technology ‒ coupled with AI platforms ‒ is being used already in Infosys’ Engine Wash VR Training. Infosys created this experience to quickly train maintenance teams in a virtual aeroplane hangar, making it possible to deliver perspectives and insights that wouldn’t be possible in the physical world.”

Concluding, Lewis comments on the limitless possibilities of the metaverse and its combination of IoT, AI and XR: “Perhaps the most exciting part of all this is that we don’t have to adhere to the normal laws of physics in the metaverse, meaning we can look at objects at any scale and from any perspective we desire ‒ something that would be completely impossible in the real world.”