Augmented Reality Revolutionizes the Driving Experience
Technology is changing the way drivers and passengers perceive the road. Augmented reality (AR) is enhancing this experience by overlaying digital information onto the real world, a concept distinct from virtual reality, which completely immerses users.
Mercedes-Benz is integrating mixed and augmented reality into its windshields.
Manufacturers like Audi, Mercedes-Benz, and BMW are incorporating AR into their vehicles. These technologies range from improved displays on infotainment systems to advanced versions of the head-up display (HUD).
A HUD typically projects information, such as speed, gear, music, or navigation prompts, onto the windshield. This lets drivers see the details without taking their eyes off the road. The newest AR displays project graphics onto a larger area of the window.
Several years ago, the Audi E-Tron SUV (now called the Q8 E-Tron) showed arrow graphics over a camera view of an intersection, indicating where to turn. Mercedes offered similar technology in its C-Class sedan around the same time. Now, these systems are becoming even more sophisticated.
The new Audi Q6 E-tron features an optional AR head-up display. It showcases information directly in the driver’s line of sight, including animations for speed limit changes, instead of static numbers.
Audi states that the forward tilt of the Q6 E-tron’s 88-inch image plane enhances the AR experience, aligning with the natural focus of the human eye. This tilt, combined with the long virtual image distance, can make the icons appear to float up to 200 yards away.
The Audi Q6 e-tron integrates video games into its head-up display.
An Audi spokesperson explained, “Due to the high image distance from the driver, the eyes do not need to refocus between the real environment and the head-up display (HUD) image. This is the biggest advantage of a HUD compared to a dashboard display. This is also a great advantage for people with corrective lenses.” Entertainment options are also integrated into the HUD. “When parked, drivers can also play games using the HUD; for example, while waiting for their vehicle to charge at a DC fast charger location. In the future, you might imagine additional entertainment tools as well,” the spokesperson said.
BMW is also expanding its user experience with new vehicle versions, set to be unveiled at CES 2025. The system provides visible displays across the entire windscreen, including for passengers, complemented by a dark screen with bright graphics at the lower edge for key information.
BMW’s Frank Weber stated, “The windscreen becomes a single large display with our new BMW Panoramic Vision, opening up completely new possibilities for the design of our vehicles. Whether the driver decides themselves which information they want to display in their own field of vision, or that all occupants can see the entire content. The revolutionary projection and the significantly more clearly structured cockpit give an impressive new feeling of space and driving. We are taking our proven ‘eyes on the road – hands on the wheel’ slogan to a new level.”
Mercedes launched its AR head-up display in 2021 on the S-Class sedan. The projection equals a 77-inch screen and integrates eye-tracking to keep the icons within the driver’s field of view. Currently, this is available on the S-Class, Mercedes-Maybach S-Class, and EQS Sedan and SUV.
The 2025 Audi Q6 e-tron boasts multiple digital displays, including the AR head-up display.
A Mercedes spokesperson said, “An augmented reality heads-up windshield (AR HUD) provides enhanced safety by projecting critical information directly onto the windshield, reducing the need for drivers to look away from the road. It also offers improved navigation with real-time overlays and can display alerts and warnings in the driver’s line of sight, making driving more intuitive and less distracting.”
Looking ahead, Mercedes is experimenting with AR glasses to enhance the driving experience, using a “Bring Your Own Device” (BYOD) approach. This would allow customers to integrate their equipment, “adapted to their eyesight and ecosystem.” The company says this offers easier integration. Linking with vehicle sensors and actuators could provide new opportunities for entertainment, wellness, and comfort, in addition to safety.
Mercedes also plans to use AR glasses for immersive virtual brand experiences, allowing potential buyers to interact with and customize vehicles. A Mercedes spokesperson explained, “Initial use cases will focus on infotainment for passengers (4D cinema experience, wellness, energizing comfort, POIs, etc.), followed by driver-related use cases (navigation, assistance, etc.). We see AR glasses as a device that extends our infotainment system and offers an enhanced visual experience for our customers. All basic functions will remain accessible without the use of glasses. As such a device, it opens new and more flexible possibilities for designing the interior of our vehicles.”
Audi is also exploring mixed and augmented reality glasses. Its Activesphere concept featured mixed reality headsets for both the driver and passenger. The user could view virtual static content that would become interactive when engaged, such as by gesture.
An Audi spokesperson concluded, “For the time being, we feel the augmented reality HUD is the best solution for drivers today as a way of safely being engaged in the driving process whilst still providing the added benefit of additional driving information.”
