When the iPhone first came out, using them seemed like an impossibility to the visually impaired. Luckily, the iPhone now provides a multitude of resources to make sure that the visually impaired have accessibility to the phone.
In fact, many blind people use their iPhone to help them navigate the world.
Microsoft has just launched an app that makes the iPhone even more accessible for the visually impaired using artificial intelligence, “Seeing AI”.
The Seeing AI platform was originally talked about last year as a prototype that only worked on smart glasses. Thankfully though, now they’ve decided to launch it to anyone with an iPhone or iPad.
In the promotional video released by Microsoft today, the company described the app:
The app narrates the world around you by turning the visual world into an audible experience.
To use the narration features, you simply have to point the camera at what you desire to have narrated to you. It will then read options for what you want the app to detect and select it.
There is a multitude of things that Seeing AI can detect. For one thing, the app can recognize friends that you have saved in your phone. Your iPhone already recognizes your friend’s faces when you take pictures of them, this works similarly. Once you ask the app to recognize your friend, you can tell the app what their name is and they will save it for the next time you need to recognize someone in front of you.
When you point your camera at that friend again it will tell you the name of your friend that they recognize, how far away they are, what their approximate age is, and what they’re wearing and what their emotion looks like. Of course, this isn’t going to be 100% accurate. It’s a guess, but it’s a guess made by AI recognition, so it’s probably not too bad.
The app also reads text aloud as it comes into view. This is particularly helpful with envelopes, documents, and signs, among other things. Seeing AI will recognize the words and read them aloud once the camera is pointed at them.
When it comes to text, the app also will guide you to make sure that you are pointing it in a way that the app can read. For books and documents, it will tell you which corners are not in the view and when to turn the page.
Seeing AI helps if you want to use cash to pay for things as well. If you hold up a bill, it will recognize the bill in your hand and let you know what you’re holding in order to more easily pay for things without the hassle of a card.
If you’re at the store or sifting through your pantry looking for a camera, Seeing AI also helps you with product recognition. The app gives you audio cues that help you locate the bar scanner on the product and then recognizes it using the bar code. You can also hear additional details about the product once it’s recognized.
The Seeing AI app offers you a way to hear narrations of other social media apps, like Twitter as well. Simply screenshot the page you’re on and then upload them into Seeing AI and the app will describe the text of the screenshot, as well as what’s going on and who said it.
The app has a largely experimental feature as well. The “scene” and “handwriting” options allow Seeing AI to guess what’s happening if you point the camera at a random scene or at someone’s handwriting, which it will try to recognize.
Seeing AI is currently only available on iOS with no word yet on when it will be released for Android. As you can see, it’s definitely an impressive app that will definitely make a difference in the lives of the visually impaired.
Just updated your iPhone? You’ll find new emoji, enhanced security, podcast transcripts, Apple Cash virtual numbers, and other useful features. There are even new additions hidden within Safari. Find out what’s new and changed on your iPhone with the iOS 17.4 update.
“Hey there, just a heads-up: We’re part of the Amazon affiliate program, so when you buy through links on our site, we may earn a small commission. But don’t worry, it doesn’t cost you anything extra and helps us keep the lights on. Thanks for your support!”
