Augmented Reality Takes the Wheel: Automakers Embrace a New Reality for Drivers
Drivers’ perspectives are evolving, with technology taking center stage in the automotive world. Augmented reality (AR) is at the forefront of this revolution, overlaying digital information onto the real world, enhancing what drivers and passengers see on the road.
AR differs from virtual reality (VR), which uses a headset to create a fully immersive environment. Instead, AR supplements the driver’s view, providing additional data without complete isolation from the surroundings. Automakers such as Audi, Mercedes-Benz, and BMW have already integrated AR technologies into their vehicles.
These systems range from enhanced infotainment displays for navigation to more advanced versions of the traditional head-up display (HUD). HUDs project information like speed, gear, and navigation prompts onto the windshield, keeping the driver’s eyes on the road.
Larger and more sophisticated AR displays are emerging, capable of projecting graphics across a wider portion of the windshield. These displays can integrate real-world objects, like street signs and road markings, as well as driver assistance features.
Pioneering AR Integration
Years ago, the battery-electric Audi e-tron SUV (now the Q8 e-tron) used AR by overlaying arrow graphics over a camera view at intersections to indicate where to turn. Mercedes-Benz introduced similar technology in its C-Class sedan around the same time.
Mercedes-Benz is adding mixed and augmented reality windshields to its S-Class and other luxury models.
Meanwhile, the new Audi Q6 e-tron SUV offers an optional augmented reality head-up display (HUD) that is significantly larger than a conventional display. It displays the same information as the screen-camera system, making it directly available to the driver’s line of sight. The system features animations that signal changes like speed limits, rather than just static numbers.
Audi has designed the Q6 e-tron’s 88-inch image plane with a forward tilt; this is intended to align with the human eye’s natural focus. Audi explains that the tilt and the large virtual image distance can make icons appear as if they’re floating up to 200 yards away.
“Due to the high image distance from the driver, the eyes do not need to refocus between the real environment and the head-up display (HUD) image. This is the biggest advantage of a HUD compared to a dashboard display. This is also a great advantage for people with corrective lenses,” an Audi spokesperson told Newsweek.
Entertainment options are also finding their way to Audi’s HUDs. “When parked, drivers can also play games using the HUD; for example, while waiting for their vehicle to charge at a DC fast charger location. In the future, you might imagine additional entertainment tools as well,” the spokesperson said.
The Audi Q6 e-tron can display video games on the head-up display.
BMW is also expanding its user experience in the new versions of its vehicles, with production versions set to be unveiled at CES 2025. Its system will offer visible displays across the entire width of the windscreen, even allowing passenger viewing. A dark screen with bright graphics along the lower edge of the windscreen will handle basic information.
“The windscreen becomes a single large display with our new BMW Panoramic Vision, opening up completely new possibilities for the design of our vehicles. Whether the driver decides themselves which information they want to display in their own field of vision, or that all occupants can see the entire content,” Frank Weber, a member of the Board for Development at BMW AG, said in a press release.
“The revolutionary projection and the significantly more clearly structured cockpit give an impressive new feeling of space and driving. We are taking our proven ‘eyes on the road – hands on the wheel’ slogan to a new level,” he added.
Enhanced Safety and New Experiences
Mercedes-Benz premiered its augmented reality head-up display on its luxury S-Class sedan in 2021. The projection equates to a 77-inch screen, using eye-tracking technology to keep the icons in the driver’s field of view. It is currently available on the S-Class, Mercedes-Maybach S-Class, and EQS Sedan and SUV.
“An augmented reality heads-up windshield (AR HUD) provides enhanced safety by projecting critical information directly onto the windshield, reducing the need for drivers to look away from the road. It also offers improved navigation with real-time overlays and can display alerts and warnings in the driver’s line of sight, making driving more intuitive and less distracting,” a Mercedes spokesperson told Newsweek.
Looking ahead, Mercedes-Benz is experimenting with augmented reality glasses to offer a new driving experience. Their approach will be “Bring Your Own Device” (BYOD). They say that this would allow customers to bring their own equipment, “adapted to their eyesight and ecosystem.” According to the company, this approach would also allow easier integration, and that linking with vehicle sensors and actuators could present new opportunities for entertainment, wellness, and comfort, as well as safety.
Mercedes also plans to use augmented reality glasses for immersive virtual brand experiences. This would enable prospective buyers to interact with vehicles and customize them. Their new mixed reality configurator uses real-time game engines and Apple Vision Pro.
“Initial use cases will focus on infotainment for passengers (4D cinema experience, wellness, energizing comfort, POIs, etc.), followed by driver-related use cases (navigation, assistance, etc.). We see AR glasses as a device that extends our infotainment system and offers an enhanced visual experience for our customers. All basic functions will remain accessible without the use of glasses. As such a device, it opens new and more flexible possibilities for designing the interior of our vehicles,” a Mercedes spokesperson explained.
Audi is also exploring mixed and augmented reality glasses. Their Activesphere concept incorporated mixed reality headsets, available for both the driver and passengers. The users could view virtual static content, which could become interactive when the driver or passenger used gestures.
“Of course, the Activesphere concept was also designed for autonomous driving, so more driver interaction could be facilitated (i.e. they didn’t need to be paying 100 percent attention to the road). For the time being, we feel the augmented reality HUD is the best solution for drivers today as a way of safely being engaged in the driving process whilst still providing the added benefit of additional driving information,” an Audi spokesperson stated.
