The all-seeing AI

Although artificial intelligence (AI) has been around for a while now, the main AI technology to reach the visual impairment market is the talking camera.

Broadly, AI means that computers interpret the information they receive and tailor their responses based on our behaviour. You may already be familiar with apps such as TapTapSee and VocalEyes, but Microsoft’s Seeing AI (currently only available on Apple devices) is a cut above the rest. Seeing AI combines the functionality of these predecessors with that of optical character recognition (OCR) apps that scan and read text aloud.

The free, easy-to-use app can scan and read formatted text, learn and recognise faces, scan barcodes to identify products, and interpret general scenes.

The range of features and its accuracy are remarkable – but what impresses me most is the simple, intuitive way in which the app is controlled. With my previous favourite OCR app, you had to hold the phone above a document and then capture it by pressing a shutter button with your other hand: no mean feat, even for the most dextrous of us. If you didn’t capture the whole document, tough. You got what you snapped.

With Seeing AI, however, you receive verbal hints that tell you whether the camera can see the entire document. And there is no need to wrestle with your phone to keep it steady while hitting the shutter button: the app automatically takes the photo for you. If you think that the scene description mode is good, then wait until you try the facial recognition setting! Not only can Seeing AI learn familiar faces, but it will also interpret moods and predict ages. This last feature could offend the more sensitive subject, but it proved quite accurate when I tried it at home (much to my wife’s chagrin).

The product scanner is a great help in the supermarket too. For me, zapping the barcode will mean the end of buying apple sauce when I want mustard, or spearmint when I want peppermint. Later versions of the app also include banknote detection, handwriting recognition, and modes to gauge light levels or identify colours. As ever, the app is not perfect, but it is rapidly becoming the must-have app of the year.

By Matt Harrison, Technology Services Manager at The Beacon Centre