Touchscreen technology rises to surface at Microsoft

Imagine a world where any surface can become a touchscreen computer: the floor, a wall, a table or even a window.

There will be no need to reach for your phone or tablet in the morning. Instead you will be able to read the news on the glass shower door while showering or look up emails in the mirror while brushing your teeth.

If the technology currently on display at Microsoft HQ in Washington is anything to go by, this could all be a reality in years to come.

In fact, the ability to turn any surface into a touchscreen computer is already possible – as long as you have a PC, a projector, and Microsoft Kinect. You can play Angry Birds on your phone, computer, tablet and TV, so why not your living-room wall too.

The Ubi Interactive touch system harnesses the technology of a projector, a PC and Microsoft Kinect to project a screen onto any surface which can then be interacted with by users.

The software-and-Kinect combo senses when a finger touches a surface, allowing the user to click, drag, drop, scroll and perform all the expected functions of a regular phone or tablet touchscreen.

The new system was developed by Ubi Interactive as part of the Microsoft Accelerator programme and is currently in beta testing.

Microsoft's skeleton detection system, as used in Kinect enabled games, isn't good enough at working out where the user is in relation to the projection, according to Ubi's chief executive Anup Chathoth, which is why custom software was created.

The software is currently available for pre-order, starting at $149, which give users the ability to project on a display up to 45-inches in size, according to Chathoth.

In the future – when playing Angry Birds on your living room wall is the norm – there will be no need to removing your phone from your pocket to calculate how to divide the bill following a group restaurant outing.

The Omnitouch system uses a shoulder mounted tiny pico projector and depth-sensing cameras, enabling users to turn their hand into a mobile phone touchscreen.

The system was developed by researchers from Microsoft and Carnegie Mellon University, and like Ubi's interactive system, could spell an end to monitors and keyboards in years to come, bringing ordinary surfaces to life.

Creator Chris Harrison says it is conceivable that anything you can do on today's mobile devices, you will be able to do on your hand using OmniTouch.

Envisioning centre

While the prototype device is quite bulky, the team believes future versions could be the size of a matchbox.

But the piece de resistance with regard to future technology is the stuff currently on display in the Microsoft Envisioning Centre, which is not yet available, but which could dominate homes and offices of the future.

Microsoft’s envisioning centre is about having computers in every surface so your home and office will be able to talk, listen, understand and help with day-to-day activities.

The envisioning centre is a working prototype that brings together big data and machine learning, combining them with touch and sense technology to create a model future home.

The centrepiece of the model home is a 120-inch 4k television, currently being used for Kinect-enabled storytelling experiences.

Microsoft doesn’t see this set-up in our homes for at least another five years, maybe even 10, and it will probably take that time for 4k display screens to become affordable.

The model home kitchen boasts a Kinect-enabled, wall-mounted display that integrates with a smart stove and counter-top to offer cooking advice.

For example, you could hold up a chilli pepper to the Kinect camera and ask it what your cooking options are. On the wall will appear a whole host of recipes and even video feeds showing how to follow them.

Once you select a recipe, the cooking instructions will appear on the countertop and stove.

"Technology is becoming more natural and intuitive to use. It will fade into the background," according to Microsoft creative director Steve Clayton.

He says the company wants to make computers so powerful and smart that they disappear.

'Magic windows'
The company's applied sciences team is currently working on this, developing a see-through 3D desktop and a magic window which hopes to combine the physical and digital worlds.

"People will be able to interact with each other in a more magical way. We want to build on that magic," Stevie Bathiche, director of research of the applied sciences group says.

The vision of the magic window is to create a a large telepresence wall that supports interactions that are close to the screen – and far away at the same time.

This would allow people from opposite sides of the world to talk and interact with one another through such a window.

They will be able to see into each other’s room with depth and perspective, and they can display objects on the “window” as well as touch and interact with them.

“We want to give people a feeling there are looking into a room and at a person face to face.”

But how do you create a screen where the participants are looking at each other, rather than a camera that in normal video conferencing is positioned above or below the screen.

Real potential

Traditional video-conferencing usually leads to a very unnatural feel to a conference but by placing cameras behind a transparent screen OLED screen, you can align the eyesight of the participants and have a conversation face to face in a very natural way.

“The computer augments physical reaction in the magic window. Computers are supposed to enhance interaction not replace it.”

Another technique that makes things feel more natural is created using a Kinect sensor to track a user’s position relative to a 3-D display to create the illusion of looking through a window and has real potential in creating realistic telepresence environments.

This technology also means people sitting side by side watching a TV can both be watching different channels. While Sony currently do this for PlayStation, you need to wear glasses to see your view of a game. In Microsoft's version, no glasses are needed.

Back in the early days of Microsoft, the company’s mission was to put a PC on every desk and in every home – an ambitious goal for the time, according to Clayton.

Behind that was the conviction that computer technology had the innate capacity to enrich peoples’ lives.

The future is also based upon this belief, of how technology can be used to make life easier and more enjoyable, sometimes in small ways and sometimes in revolutionary ones.