BMW turns inward for CES, announcing new gaming, streaming, AR, and AI features

After back-to-back years of bringing unique, compelling concepts to the floor of the Consumer Electronics Show, BMW is turning inward in 2024 and focusing on the interior of the vehicle — namely, the types of technology and media content the automaker thinks its customers will want to interact with as they drive their high-priced, high-powered vehicles into the future.

In a statement, Frank Weber, chief technology officer and member of the board of management responsible for BMW Group development, said that the company was showing “more content, more customization, and more gaming.” And coming soon, the company’s operating system will feature “perfectly aligned” augmented reality and “strong, reliable” AI to enable better communication between the car and the driver.

Practically, this means that BMW is bringing more streaming video content and gaming options to its vehicles. For its future lineup, the automaker insists that its customers will be able to use previously unavailable technologies, including augmented and mixed reality and in-car voice assistants powered by generative AI. No barf bags required.

BMW Operating System 9, also known as iDrive 9, relies on Google’s Android Automotive software. Customers who purchase the optional “BMW Digital Premium” package will get access to a range of third-party apps from the company’s ConnectedDrive store, including gaming on their vehicle’s infotainment display while parked or charging at an EV station. Assumedly, rear passengers will be able to play as well using massive screens such as the BMW i7’s 32-inch drop-down display.

And now, for the first time, BMW owners will be able to use real video game controllers when playing. The automaker says “any standard gaming controller with Bluetooth capability” can be used as long as the customer’s vehicle supports Operating System 9. Over-the-air software updates that will enable video game controllers are coming in 2024, BMW says. One such game that can be played using controllers is Beach Buggy Racing 2, a mobile game available in both the iOS App Store and the Google Play Store.

BMW is now the latest automaker to bring gaming to its cars. The trend was kicked off by Tesla, which has gone faster and further (sometimes too far) in offering gaming options to its customers. Tesla customers can use game controllers or even the car’s own native controls, like the steering wheel, to play certain games.

Using the company’s preexisting AirConsole app, vehicle owners can choose between single-player and multiplayer games. “Casual” games in which customers can use their smartphones as controllers will also be available.

Of course, in-car distractions are at an all-time high, and BMW will have to put guardrails in place to ensure that vehicle owners aren’t able to misuse these features while driving or that passenger gaming isn’t distracting the driver.

Video streaming

If you’re not a hardcore or even casual gamer, perhaps you’ll find BMW’s expanded video streaming options more up your alley. The company is updating its in-car video app for vehicles running Operating System 8.5 and 9, including on-demand and live offerings. BMW is bringing these expanded options to more vehicles in 2024, such as the BMW X2 and all models in the new Mini lineup.

BMW’s i7 sedan — the one with the enormous 31-inch drop-down movie screen for rear passengers — includes embedded Amazon Fire TV. The company is now working to integrate something called “DTS AutoStage Video Service powered by TiVo,” which it says includes “both live channels and on-demand media libraries covering the categories news, sport, entertainment for kids, movies and TV series.” That system was first launched with BMW’s 5 series and will be coming to more “executive and luxury” models via an over-the-air software update in 2024.

Augmented and mixed use reality

Automakers have been trying to make AR applications for their cars for a very long time. But now, BMW is claiming to be among the first to get it right.

The company says it will demonstrate this capability “for the first time” at CES in Las Vegas. (Our producer Becca Farsace has heroically volunteered to try it out at the show, so look out for that. Prayers up she doesn’t get too nauseous.)

Passengers will be invited to don a pair of Xreal Air 2 AR glasses, which use Micro OLEDs to project a 1920 x 1080 pixel display in front of your eyes. In this way, they will “see how navigation instructions, hazard warnings, entertainment content, information on charging stations and supporting visualizations in parking situations are embedded perfectly into the real-world environment.”

How perfectly? TBD. You’ll recall that BMW is also working with Meta to alleviate any issues of interacting with virtual objects while moving through the real world at high speed. Normally, AR and VR applications would have an extremely difficult time working in a moving vehicle, but BMW says it has figured it out. (I think it’s worth quoting the company in full so you can properly understand the level of confidence we’re talking about):

The BMW Group has been working on the development of AR/MR technology since 2008 and investigating possible applications for head-mounted displays in research projects. The BMW engineers, together with their development partners, have now succeeded in showing stable augmented reality and mixed reality content to passengers in a dynamically moving vehicle for the first time. The displayed image remains securely embedded in the environment even in challenging driving situations, such as when turning, driving over large bumps or accelerating. To achieve this, the tracking system of the AR/MR device in question has to be linked to the vehicle’s sensor data.

BMW says it wants to create an “industry standard” for how AR and MR can be used in vehicles, pointing to its work with digital keys as an example of how it would approach such work.

The company assumes that rear passengers, such as luxury vehicle owners who are often chauffeured to work, will want to wear Xreal’s AR glasses to do work or participate in virtual meetings. Gaming is another possible application.

Generative AI and LLMs

BMW says it is making progress on using Amazon’s Alexa technology to build its own custom voice assistant, which it first announced back in 2022. The company says it’s ready to show off its progress so far in advance of “a potential series rollout.” (In other words, still under development and no guaranteed product to announce yet.)

Using Alexa’s large language model as its foundation, BMW’s Intelligent Personal Assistant will be able to respond to “quick instructions and answers about vehicle functions in a much more human, conversation-like manner, while at the same time being able to control some vehicle functions.”

To be sure, Alexa has been available as an in-car voice assistant in BMW vehicles since 2018. But BMW says its current lineup lacks “complex processing capabilities” and “human-like interactions and dialogs,” but that could be coming soon thanks to LLMs.

Automakers across the industry are scrambling to create stronger links between their voice assistants and core vehicle functions. What’s interesting here is that BMW is sticking with its plan to use Amazon’s Alexa as its basis rather than shifting to Google’s AI products, which could theoretically complement the automaker’s Android-based operating system.

Teleoperation

Lastly, BMW says it’s working with auto supplier Valeo on an “automated valet parking” feature that also could work via teleoperation.

According to the automaker, customers would leave their vehicles in designated drop-off areas. The vehicle would then search for a vacant parking space itself, maneuver into the space, and later drive back to the pick-up area for collection by the driver. “Either an autonomous driving system or a teleoperated valet system can be used as the underlying technology for this,” BMW adds.

The challenges of teleoperation are myriad, mainly around maintain a strong connection between the remote operator and the vehicle without any latency. Most autonomous vehicle companies include some form of teleassistance, but full-blown remote operation remains a mixed bag.

Like most automakers, BMW has been working on self-parking technology for a bit, even getting recognition from the US Department of Transportation for it at one time. But self-parking is tricky, and while all companies have some self-parking or even automated parallel parking system available, it would be a stretch to suggest that it’s entirely bug-free.