Seán Ó Faoláin’s famous quote – “There is only one admirable form of the imagination: the imagination that is so intense that it creates a new reality, that it makes things happen”– came to mind recently as I mused about artificial reality technology.
In 1957, Morton Heilig, a Hollywood cinematographer, observed that only a small proportion of a spectator's field of view, and only in two dimensions, is on a cinema screen. Could there instead be a three dimensional (3D) picture, filling all of the spectator's field of view, along with stereophonic sound?
Heilig failed to attract investors, but scraped enough funds together to build his ‘sensorama simulator’ prototype. The machine projected images taken simultaneously from three 35mm cameras strapped on to a cameraman.
You placed your face into a recessed alcove in the machine, to watch a three-dimensional recording augmented with stereo sound. Thus artificial reality was born.
In 2009, 17-year-old Palmer Luckey made a handy income fixing and reselling damaged iPhones in Long Beach, California. Having an intense interest in electronics, he self-financed his prototype of a head-mounted display, with 3D stereoscopy, a 270-degree field of view, and stereo sound.
In 2012, he ran a Kickstarter campaign to raise funds from future customers for a commercial version of his virtual reality (VR) headset, raising nearly 10 times his target. Two years later his company, Oculus VR, was acquired by Facebook for $2 billion.
Panoramic view
Today, VR headsets are relatively common and affordable. Whilst wearing a headset, you can move your head in any direction to see the imagery surrounding you and to identify from where sounds are coming. Being immersed into a panoramic view of reality can be exhilarating, and occasionally disturbing: perched on the flanks of Everest, or face to face with a tropical shoal above a coral reef, or high above Earth in a spacewalk.
VR content also includes synthetic worlds. Digital animation, particularly for computer gaming, creates artificial worlds which yet may appear so intense that – in Ó Faoláin’s words – you feel a new reality. The fear of the unseen – what may lurk behind you, or what may be in the impenetrable gloom – can literally terrify.
But there is something sinister about being digitally blindfolded in an opaque divers mask-like contraption. Like the first mobile phones, VR headsets resemble bricks, but now strapped to your face rather than then held in your hand. The iPhone revolution made mobile phones a fashion item. We need a new Steve Jobs to do so for headsets.
Exploration in VR is factitious. You may well want to walk into or away from a view. Yet if you do so while blindfolded, you are likely to trip over a piece of furniture in the real world.
There thus is considerable investment and research efforts in ‘augmented reality’ (AR) in which your real-world view is enhanced by digital content. You can see the digital images but still avoid the furniture.
AR is a larger market opportunity than VR since it provides a mechanism to digitally label objects and add digital inserts into views of the real world. Technicians can be guided to specific machine parts, student doctors can see internal organs, and architects and their clients can explore 3D models projected on to a table top.
Global craze
Smartphones and tablets can be used for AR. The real-world view seen in your smartphone camera can be overlaid with digital inserts, as was illustrated by the global craze for the Pokémon Go game in July 2016.
Other examples include viewing potential furniture purchases in situ in your home; browsing paint schemes and optional extras in a car showroom, or digitally trying on clothing and accessories in a fashion store.
A challenge in AR is to maintain proportion and position of each digital insert on to the real-world view. As the viewer moves closer or further away, the insert should scale proportionately. If you move around the view, then the sides or back of the digital artifact should be seen, and in 3D.
In contrast, low-fidelity AR simply presents each insert as a fixed, flat, two-dimensional image, without depth and proportion.
Maintaining the proportion of each AR insert on to the real-world view is technically not overly difficult if the insert is just a synthetic digital image.
However, the more intriguing possibility is to create AR inserts which are images or video taken in the real world, and still maintain proportion and scaling in AR.
Rather than having to slap a VR brick across your face to see a video recording of a tropical shoal swimming above a coral reef, AR may soon make it possible to see the tropical shoal swimming above your coffee table in your own sitting room. Or Johnny Sexton’s winning kick in 3D on your kitchen countertop, viewable from all sides as you move around the counter.
Live, rather than recorded, real-world AR inserts would be even more intriguing. Imagine being able to converse live with a 3D insert of your sister in Australia, or your business colleague in Tokyo, or your instructor in Wisconsin, each sitting beside you wherever you happen to be.
