Virtual Immersive Meetings

Posted by Encora on Jun 16, 2023 12:26:00 PM
The Data Hub solution automated resource-intensive and complex tasks. This automation enabled the client to focus on exploring and analyzing their data for improved commercial strategy.

Topics: Digital Experience, Metaverse

Industry Context

After the Pandemic, there was a significant change in the workplace model. Most organizations adopted a "hybrid" work approach, where employees worked remotely for about 90% of their time. This shift led to a significant challenge in terms of collaboration for work. Most organizations leaned on virtual meeting tools like Teams and Zoom for virtual synchronous employee interaction. But these tools do not enable immersive interactions. Our Virtual Immersive Meetings idea was born from this need for immersive virtual collaboration among employees for project meetings, brainstorming, and learning workshops. Any organization can leverage this experience and it is industry agnostic. 

Immersive Meetings 1

Idea

The core of the idea is to create a virtual environment that simulates a physical meeting/conference room. The virtual environment provides the participants with an immersive experience of participating in a virtual discussion, with the ability to present, brainstorm, and communicate with the participants. The participants are represented by an avatar in the virtual meetings. The participants can move freely in the meeting space and interact with other participants just as they would in an actual physical meeting room. 
Immersive Meetings 3

Challenges

  • Asset Creation (Environment & Avatar): We planned to create a 3D environment with minimum details optimized for virtual space while ensuring all the details. We faced this challenge using PBR (Physically Based Rendering) texture maps. Our next focus was the Avatars. We designed low-poly 3D avatars which needed variation in appearance; therefore, we created various hairstyles and textured them with a different palette of colors. We encountered a challenge while rigging the head of the avatar. Consequently, we devised a distinct head structure for the avatar that the user can dynamically control using their controls 
  • Lighting: We used a small combination of lights to manage the runtime processing load and deliver a seamless experience. Unity bakes these lights, performing lighting calculations in advance and saving the results as lighting data, which the team will then apply during runtime
  • Sync in Avatar Interactions: We used photon servers that simultaneously manage numerous connections to support multiple users with different avatars. The avatar's position and orientation data are converted from Vector3 to bytes and transferred via the network to reduce the lag. Network models limit bonds to a range of 20 – 2,000 concurrent users
  • Voice Chat: We implemented voice chat with speech indications on top of the avatars to visually represent the person/avatar speaking in the virtual space. The echo occurred when multiple avatars started talking simultaneously, which posed a challenge. Despite the photon framework handling it, we had to tweak the configurations in the photon to resolve the echo in the voice
  • App configuration: We set up three different App IDs in photon to distribute the load for the app configuration. One App ID is dedicated to character movement and the whiteboard, another is for voice chat, and the final is for text chat 
Immersive Meetings 2

Results

We have designed an experience in the metaverse that allows us to scale up environments based on the host's selection, making meetings resemble physically attended meetings. The host can secure entry to the meeting with a passcode. We can expand the number of members per meeting without modifying any code handled outside the application within the photon's dashboard. The host can also dynamically load video streaming per the meeting's agenda through simple clicks. Before entering the meeting room, all meeting members will have those videos automatically loaded. 

The meeting room environment limits teleportation to specific areas, such as the table, to prevent overlapping avatars with the background. The whiteboard canvas areas are designed to extend to any surface within the environment, and any character can be made writable by tagging in Unity. However, there may be a slight lag in the application due to the user's location and network speed, which can cause a delay in movement and interaction with others in the virtual meeting space. Our team of experts uses Generative AI to enhance digital products for multiple clients, delivering an unmatched experience. 

 

Immersive Meetings 4