The Role of Augmented Reality Audio

In March 2018, Bose did something unexpected. They released a new wearable prototype at SXSW: 3D printed, augmented reality audio sunglasses. Instead of adding artificial visuals, they focus on sound-based augmented reality (AR), using motion sensors with GPS information from the user's smartphone to trigger audio cues. This is a new take on what we've come to expect from AR.   

Sound has always been a fundamental part of our lives. It adds more texture, context and meaning to our daily interactions and immerses us. Yet, while sound serves as one of the dimensions we look for in today's movies and video games, virtual and augmented reality sound it is often overlooked.

Why is Virtual and Augmented Reality Audio Overlooked?

The development of AR/VR begins in optics. Visual immersion is priority for developers as they work toward overcoming field of view (FOV) limitations with existing devices. After all, visuals typically serve as the main branch of the AR/VR experience. 

Then there are technical design challenges. Our eyes are away from our ears, which presents unique obstacles to tackling device design. Today's wearables typically have a horizontal fit over the eyes and come in bulky form factors. Extending them to fit the ears would expand the vertical reach, thereby adding to the bulkiness of the device. 

When we shift the focus from wearables to smartphones, we run into a new set of challenges. In their current form, smartphones only provide a 'flavor' of AR/VR. In either case, sound is experienced directly through the device speakers or through an additional set of headphones. In the case of virtual reality, the user could be looking at three different form factors to immerse themselves: the smartphone, headset and headphones. And even then, the experience can still be somewhat clunky. 

Finally, there are battery concerns with untethered devices. The processing power required for AR/VR visuals considerably impacts the battery life of the device. Adding acoustics that match the visuals can further affect battery life, leading to less-than-desirable user experiences. Combined with the high cost of manufacturing today's gadgets, it is easy to see why virtual and augmented reality audio may not be a priority in development.   

But can AR/VR be truly immersive without sound? Some creators claim sound makes up 50 percent of the experience. Without the right audio cues to match the visuals, the brain doesn't comprehend AR/VR with full fidelity. Poor quality or the lack of sound takes away from the "event," crippling the stories being told through these mediums. 

The Role of Virtual and Augmented Reality Audio

AR and VR are designed to deliver unique experiences, with benefits beyond what's available to us today. The next generation of devices will free our hands to interact with our realities, making us more productive. Here's how the proper use of acoustics can enhance existing devices:

As Lars Runge, Jabil's vice president of engineering design mentions, several of our factories and customer experience centers already utilize AR in manufacturing. In these interactions, AR headsets serve two purposes: overlaying information onto the user's physical world and assisting users via voice commands. Fully implementing sound into the device allows two-way communication between an employee who may need help troubleshooting and his supervisor. In other instances, audio provides an artificial intelligence voice assistant that can answer simple questions about the task at hand. Either way, the presence of good sound technology makes employees more productive.

In a VR game, Hi-Fi sound makes the experience feel like reality. Through spatial audio, players are given cues to look behind them or explore their surroundings, creating more immersive experiences. In addition, the systems can be leveraged to provide person-to-person or person-to-machine communication, unlocking new features and use-cases within the game. 

Improving AR/VR by Improving Acoustics

As companies navigate the path of AR/VR, they need to make virtual and augmented reality audio a part of their long-term strategy. While there is no clear device-of-choice for these experiences, creating one that incorporates sound naturally is a competitive differentiator. Here are some considerations to enhance AR/VR with acoustics:  

• Integrating Hi-Fi sound to VR headsets
• Adding a separate microphone for feedback mechanisms 
• Developing separate, low-power earpieces that connect to your devices 

It took movies more than three decades after their invention to integrate sound as part of the experience. Up until that point, they had been silent as the technology to combine sound and visuals didn't exist. In an effort to immerse viewers deeper into the art, movie theaters hired musicians to accompany what they saw on screen. While we may be light years ahead in technology, it feels like we're at a similar point with AR/VR. We have a band-aid approach.

Whether it's through the evolution of current AR/VR devices or extending the limits of existing technology, product companies must think of the overall user experience. Until augmented reality audio technology advances significantly, promises of true "immersion" will fall on deaf ears.