Screens have proliferated in our cars like pigeons flocking to a dropped sandwich. Most modern cars now come with two screens – one for instruments and one for infotainment – while many high-end models now offer the option of an extra screen for the front-seat passenger. Combine that with the screen on your phone and the stage is set for a whole world of distraction.
If all of the screens migrated to the windscreen, would that make things safer?
That’s what Apple is aiming for with a patent it has just filed in the United States. In the patent, Apple speaks of an “augmented reality display system which generates a display on one or more transparent surfaces of the vehicle”.
We’re already well used to the idea of a head-up display (HUD) in a car. Having been initially used on very high-end models, they’ve now proliferated to affordable cars. The idea came from fighter jets in the 1960s – by projecting graphical information about the aircraft’s speed and position directly on to the windscreen, it meant the pilot could keep their head up and looking out, rather than peering down at instruments. In cars, the idea is basically the same – by projecting your speed, your sat-nav instructions and other information directly in front of your eyes, you’re always looking through your instruments, rather than having to look down into the cabin. Thus, your attention can be kept more on the road ahead.
Apple’s idea is basically to take that concept and expand it upwards and outwards. The company’s CarPlay software – which allows you to mirror your phone’s screen on your car’s touchscreen and allows limited, road-safety-sensitive access to apps such as music streaming and navigation – has become ubiquitous, alongside Google’s rival Android Auto set-up. Indeed, so keen are we on Apple CarPlay that BMW customers revolted against the company’s plan to introduce an annual fee for using the system.
The augmented reality windscreen would take CarPlay and, to an extent, put it right in front of your eyes. Augmented reality in-car systems already exist, up to a point. Mercedes, for example, ties its sat-nav into the forward-facing camera in the windscreen, and can overlay directional arrows on to a live picture of where you’re driving, making sure you don’t miss a crucial turn. Apple’s augmented reality windscreen would put that front and centre in your line of vision, along with other crucial information such as your car’s speed.
The patent application also includes reference to “display elements which indicate a position of environmental objects which are obscured from direct perception through the transparent surface”. That could mean no longer losing sight of a cyclist or pedestrian when they’re obscured by your windscreen pillar. Or it could mean that the system can highlight a speed limit sign that has become obscured by overhanging vegetation. It could maybe even mean that the system can work out what’s ahead of you in thick fog, using the car’s forward-facing radar.
There are other potential uses. Apple is working on a system that, if you happen to be speeding, can project images of speed bumps in front of you, and even use a car’s adaptive suspension system to simulate driving over those illusory bumps, getting you to slow down. Another part of the system claims to detect “human child individuals” – which sounds a touch creepy and sci-fi – and warn the driver if one or more is about to run out from behind parked cars.
All of this sounds really cool, but there are some unanswered questions. Apple doesn’t specify if this system will use a projector, shining info on to a conventional windscreen. If it does, does the car have to have the projector fitted when it’s built? Or will Apple sell aftermarket conversions?
How do we know we’re looking at the real world if we’re looking at it through an Apple-curated screen?
If it’s not a projector, will a special LCD screen have to be used? And if so, how much is that going to cost? Worse, how much would a broken one cost to fix?
There is also a slightly worrying element. In the patent’s abstract, Apple says that the system can “include display objects which conform to environmental objects and can obscure and replace content displayed on the objects”. That sounds a touch Orwellian, as it implies Apple can take things in the real world – say, an advert for a rival phone or laptop – and replace it with an image of one of its own products.
How do we know we’re looking at the real world if we’re looking at it through an Apple-curated screen?
Equally, when it comes to regulations surrounding the use of full-windscreen displays, there are few strict regulations regarding how much can be displayed, and whether or not the displays cross the border between information and distraction. That may yet be a challenge to future HUD designs as regulators are bound to become concerned about such distraction.
Apple isn’t the only company looking at this kind of technology. Earlier this year, in its i-Vision Dee concept (an early look at the upcoming all-electric Neue Klasse saloon due to launch in 2025) BMW showed off a five-stage augmented reality system that does away with dashboard screens altogether and instead uses projection on to the car’s windscreen.
“Drivers can decide for themselves how much digital content they want to see on the advanced head-up display,” said BMW. “The five-step selection ranges from analogue, to driving-related information, to the contents of the communications system, to augmented-reality projection, right up to entry into virtual worlds. In parallel, dimmable windows can also be used to gradually fade out reality. ”
In the BMW system, the entire windscreen isn’t a display, but the full width of the lower part of the screen is. The tech is still at an early development phase, but the Munich company says that it will be introduced in that 2025 Neue Klasse.
