When Amnon Shashua’s aunt began to lose her sight, she asked her nephew to find a solution.
He was better placed than most to fulfil that request: a professor specialising in computer vision, he had already co-founded a company with Ziv Aviram called Mobileye, which developed vision-based advanced driver-assistance systems focused on avoiding collisions. It was bought by Intel last year in a $15.3 billion deal. Shashua still heads up the division within Intel; Aviram has since left Mobileye.
In 2010, the pair founded OrCam. Its team – 120 software, hardware and algorithm developers – began to work on a single product, which eventually became MyEye. The device allows someone with a visual impairment to access any printed text on any surface wherever they are, using a combination of a camera, advanced computer vision and machine learning, and a small speaker.
It was an obvious application for computer vision, explained Leon Paull, OrCam's head of business development. "It's quite unique in the assistive tech arena to have that level of investment and effort in one company behind one product," he said.
Photographs
The device works in a very simple, gesture-based way. The wearer points to whatever it is they want to “see” or read, and the MyEye device takes a photograph. Sophisticated software processes what is in front of it and then, in the case of text, reads it back to the user. What’s more, it can also recognise friends’ faces so it can alert the user as they approach.
The small speaker is placed in such a way that it is easy for the wearer to hear the output without it being overly intrusive.
Privacy is an area where OrCam is treading carefully. As it stands right now, there is no facility to connect a web-based service directly to the device and allow it to automatically `learn' your friends' faces
The company is on to the second generation of its devices now. The original OrCam MyEye device launched in 2015. That device consisted of the camera and speaker unit, which attached to the wearer’s glasses, and required a pack about the size of a mobile phone that was wired to the camera and contained all the computing power and batteries the system needed. The pack was about the weight of a mobile phone, and a little thicker.
The latest version, the MyEye 2.0, dispenses with the wires and additional pack. Instead, everything is contained within the unit that attaches to the wearer's glasses, and a lanyard clips on to prevent the user losing the device. At £3,500 (€3,974) for the new generation of the device, a bit of extra security is probably a good idea. The company made a splash at last month's CES exhibition in Las Vegas, where it walked away with the Last Gadget Standing award.
MyEye means you can read a newspaper, find products in a shop or tell who is approaching you without the need for outside assistance. Giving people with vision impairments back a degree of independence is something of which OrCam is proud.
“It gives them a degree of independence that either they have never had or they had and lost,” said Paull. “It gives them the ability to once again sit at home and read a book or the newspaper, sort their post, read the menu in a restaurant, stay at work because they can read the documents around them.”
MyEye can recognise colours and products, read newspapers and restaurant menus, and decipher street signs. If the text is large enough to distinguish, MyEye can read it. Depending on where you buy the device, it will also recognise banknotes, an area where, Paull said, many users would otherwise run into difficulty. The version of the device sold in the United States will recognise different dollar amounts for both US and Canadian currencies; the one sold in Ireland will distinguish between the denominations of euro notes.
The beauty of the MyEye, though, is that it learns. Whether it is products or people, the device can add to its own internal database. You can teach the device faces, with a couple of clicks, and next time that person is in front of you, MyEye will tell you which of your friends has come into view. If it doesn’t know, the device will tell you if the person in front of you is male or female.
That has presented its own set of challenges. Artificial intelligence may be the way of the future, but it isn’t infallible, as OrCam found out. In addition to the facial recognition that could determine if it was a man or a woman standing in front of the camera, OrCam also – briefly – enabled its software to determine age.
“We’re pretty reliable on the gender,” said Paull. “We did have age estimation but it wasn’t quite reliable enough and we said until it was 100 per cent perfect we’re not going there.”
Dyslexia
MyEye is not just for people with vision problems, though. There is a version of the product that can be used by people with dyslexia. That software doesn’t have the AI component that learns with the user, because it doesn’t need to carry out tasks such as facial recognition or product recognition.
At the moment, MyEye is still considered a niche product. But it is possible that, in the future, there will be a consumer application. Imagine never forgetting another name because the small camera clipped to your glasses instantly whispers the name in your ear, drawing information from your email about when you last spoke.
That creates as many issues as it solves, though, chiefly around data protection. Facebook found itself in hot water with privacy watchdogs over its facial-recognition software in Europe, with the ultimate result being the removal of the feature and the deletion of all associated data in the European Union.
Privacy is an area where OrCam is treading carefully. As it stands right now, there is no facility to connect a web-based service directly to the device and allow it to automatically “learn” your friends’ faces, although the second generation of MyEye makes it technically possible.
“Privacy is a big issue, especially for people with a visual impairment – they are very vulnerable. So they do have concerns, which is why we’ve maintained throughout the development of this product that it be self-sufficient in the sense that it doesn’t rely on the internet for any of its operations,” said Paull.
MyEye is not reliant on the web or any cloud services either: all data is processed on the device. Images it takes are not stored or sent anywhere; the only exception is biometric maps of faces you ask the device to learn, which are stored on the device to allow it to recognise faces in the future. It may use the internet for augmented functions and software updates, but the core functions will remain on the hardware.
Aside from the obvious data privacy implications of relying on web services, there was one very obvious reason why OrCam chose to keep everything MyEye does on the device: reliable access.
“When you are using it as a daily assistant you can’t rely on net connectivity. You also don’t want the delay that is typically inherent to an internet-based service,” said Paull, saying that would lead to frustration and eventually using the device less.
