OrCam: A New Vision for Machine Learning
Can machine learning help people see again? OrCam says yes.
In late 2013, Israeli computer scientist Amnon Shashua and entrepreneur Ziv Aviram introduced the OrCam MyEye, a wearable visual assistant 鈥渟ystem鈥� 鈥� part computer, digital sensor, speaker and machine learning algorithm(s) 鈥� aimed at improving the lives of the visually impaired, a population numbering more than 20 million in the U.S.[1] and over 285 million worldwide[2].聽These include people afflicted by medical conditions such as macular degeneration, cataract and diabetic retinopathy as well as others who have suffered vision loss in military combat. The device is about the size of your finger, weighs just under an ounce and costs $2,500, roughly the price of a good hearing aid.[3]
Design Spec(tacle)s:
While wearing the MyEye attached to one鈥檚 glasses, a user points at whatever s/he wants the device to read and the device鈥檚 camera, upon recognizing the human鈥檚 outstretched hand, takes a picture of the text 鈥� be it a billboard, food package label or newspaper 鈥� and, after running the image through its algorithms, reads the text aloud.聽 Leveraging supervised learning technology, the product is trained on millions of images of text, products, and languages so that it can identify and interpret the proper image when it comes into view.
While OrCam began as a 鈥淎I-native鈥� technology, advancements in machine learning have pushed the boundaries of what the MyEye is able to do 鈥� and created challenges for Shashua and Aviram as they seek to further develop their product.
Seeing Around Corners:
Existing product improvements: OrCam has capitalized on ML advancements to improve its core product. MyEye benefits from greater image capture capabilities as the underlying software is continuously trained on new formats of text such as new fonts, sizes, surfaces and lighting conditions.聽 It can now identify whether there is sufficient natural light to capture an image and can recognize when an image is upside down and clue the user to flip the item.聽 In addition, MyEye can announce color patterns (useful for dressing oneself in the morning), recognize millions of products and store additional objects like credit cards or grocery items.
New product features: Beyond text, OrCam has focused on adding facial recognition capabilities, which similarly harness the power of machine learning. 聽Through supervised learning, users can record family members faces in <30 seconds[4] and the device will cycle through its programmed dataset to identify this person the next time s/he comes into view.聽 Further, the device can parse unstructured inputs (e.g., new faces) to give the user clues about bystanders it doesn鈥檛 recognize (鈥渋t鈥檚 a young woman in front of you鈥�).[5]
Challenges for 20/20 and Beyond:
One of the classic tensions with wearable technology surrounds how much intelligence is stored on the device versus in the cloud. 聽On the one hand, the device should be aesthetically light and easy to use (the initial product was burdened by a clunky cable and base unit), but on the other, it should be “big” enough to process, store and compute a lot of data. One option to overcome the constraint of limited built-in memory is using real-time cloud storage, but doing so consumes a large amount of power and drains the battery (today鈥檚 battery life is ~2 hours[6]), not to mention raising a host of user privacy issues if the device is synced with other personal, cloud-based apps.
How better to solve this problem than by turning ML capabilities inward to make the product itself function more efficiently?聽 For example, the MyEye can harness machine learning to decide when to process new images (defer processing when in 鈥渓ow battery鈥� mode), which ones to process (prioritize those with sufficient light exposure) and how to regulate its overall database (delete redundant images or re-capture a low-resolution image). [7]聽 In addition, by training its image sensors to re-aim when an object is only in partial view, the MyEye will be able to process information more accurately and, importantly, efficiently, as it no longer wastes time matching incomplete data. [8]
Future Optics:
One key question for OrCam surrounds how far it should expand its target market. The basic product design 鈥� which required a user to point to an object to trigger the reading system 鈥� was geared for the 鈥渓ow vision鈥� market; in fact, it precluded 鈥渇ully blind鈥� individuals who wouldn鈥檛 know where to point. 聽However, much more potential exists to embed the product into daily life for the visually impaired and beyond.聽 What if OrCam could leverage its accumulated dataset to send corresponding information about an individual 鈥� such as name, birthdate, or last time of meeting 鈥� once it recognizes that person鈥檚 face?[9]. 聽Or, if the device reads McDonald鈥檚 labels every Sunday, can it make statistical inferences to recommend coupons associated with these user preferences? Overall, how should OrCam weigh increasing the 惭测贰测别鈥檚 functionality and mass market appeal against protecting user privacy and maintaining a consumer-friendly, wearable design?
(Word count: 798).
Citations:
[1] Erik Brynjolfsson and Andrew McAfee. 鈥淭he Dawn of the Age of Artificial Intelligence.鈥澛�The Atlantic (February 2014).
[2] 鈥淕lobal Data on Visual Impairments.鈥澛�World Health Organization (2012).
[3] Katharine Schwab. 鈥淭he $1 Billion Company That鈥檚 Building Wearable AI for Blind People.鈥澛�Fast Company (May 2018).
[4] Alex Lee. 鈥淢y Eye 2.0 uses AI to help visually impaired people explore the world.鈥澛�Alphr (February 2018).
[5] Romain Dillet. 鈥淭he OrCam MyEye helps visually impaired people read and identify things.鈥澛�TechCrunch (November 2017).
[6] Katharine Schwab. 鈥淭he $1 Billion Company That鈥檚 Building Wearable AI for Blind People.鈥澛�Fast Company (May 2018).
[7] Yonatan Wexler and Amnon Shashua. 鈥淎pparatus for Adjusting Image Capture Settings.鈥� Patent No. US 9,060,127 B2. United States Patent Office (June 2015).
[8] Yonatan Wexler and Amnon Shashua. 鈥淎pparatus and Method for Analyzing Images.鈥� Patent No. US 9,911,361 B2. United States Patent Office (March 2018).
[9] For HBO鈥檚 Veep fans, think of this as a real-life Gary Walsh to our Selina Meyer!