I haven’t tried it for myself just yet since this is the first I’ve heard of it, but if Microsoft’s Seeing AI app works as advertised, holy shit!
Seeing AI, a free app that narrates the world around you, is available now to iOS customers in the United States, Canada, India, Hong Kong, New Zealand and Singapore.
Designed for the blind and low vision community, this ongoing research project harnesses the power of artificial intelligence to open up the visual world and describe nearby people, text and objects.
The app uses artificial intelligence and the camera on your iPhone to perform a number of useful functions:
- Reading documents, including spoken hints to capture all corners of a document so that you capture the full page. It then recognizes the structure of the document, such as headings, paragraphs and lists, allowing you to rapidly skip through the document using voiceover.
- Identifying a product based on its barcode. Move the phone’s camera over the product; beeps indicate how close the barcode is – the faster the beeps, the closer you are – until the full barcode is detected. It then snaps a photo and reads the name of the product.
- Recognizing people based on their face, and providing a description of their visual appearance, such as their gender, facial expression and other identifying characteristics.
- Recognizing images within other apps – just tap Share, and Recognize with Seeing AI.
In addition to full documents and barcodes, it will also be able to read things like signs and labels, which if done well could be a pretty big step up from what the still awesome and useful TapTapSee does now. Oh, and it will even try to describe any picture you take in detail, a handy feature for anyone who has ever let a sighted friend borrow their phone or had one take a photo for you only to discover that they actually took 12 of them.
And remember, all of this is free. Maybe it’s only free because it’s a research project, but if it’s going to lead to greater accessibility in all sorts of mainstream applications down the line, who cares?