AI opens new worlds for the blind
When Chris Donahue opens the door to let his dog out, he is unable to see where the animal runs.
The Brighton Heights resident is blind, but he can find his dog outside via the Ray-Ban AI (artificial intelligence) Glasses he’s wearing.
“Meta, I am blind,” says Donahue, 48. “Do you see my dog?”
A voice responds through the glass frame on Donahue’s ear — “There is a dog in the middle of the yard chewing on a stick.”
The voice comes from inside the speaker in the glasses through a connection to the Meta AI app on his phone via Bluetooth. A voice command activates a camera to take photos and describe the image. The glasses can be used to describe the labels of what is on a shelf in a cabinet, relay microwave directions and the product name on a box and can also be used to take videos, among other things.
The glasses look like regular glasses and cost around $300. The glasses have a limited battery life of around four hours.
“This is the best invention I have ever seen,” said Bruce Story, access technology instructor at VisAbility Pittsburgh, who works to teach the latest technology to clients.
“It’s a game-changer,” said Erika Petach, president of VisAbility Pittsburgh, formerly Blind & Vision Rehabilitation Services of Pittsburgh. “Smartphone apps help create a more accessible world. Our clients can do anything we can do, except drive a vehicle.”
But that, too, is changing with the introduction of self-driving cars, some of which have been tested in Pittsburgh.
Independence
The goal is independence, Petach said, and to “focus on everyone’s abilities, not their disabilities.”
AI can assist people who are blind and have vision loss. Although not perfect, it is continually being revised — like most technologies.
But not everyone is on board right now.
“I am a little apprehensive about this stuff,” said Tim Miller, executive director of Westmoreland County Blind Association.
Miller, who is legally blind, said the technology is so new that it will take time to see how it improves and that it isn’t specific to those who have difficulty seeing — anyone can use the glasses.
There isn’t technology — yet — to replace how people’s eyes connect to their brain said Miller, who is intrigued by Elon Musk’s Neuralink brain chip implant, which is currently in clinical trials to help paralyzed patients by connecting their brain to a computer.
In April, Musk announced that the technology would be used as a way to restore vision in people who are blind by the end of the year.
“These things take time,” Miller said. “It’s a supply and demand kind of thing. If more people in the world were blind, there would be more time and money invested in this technology. It is all good stuff, I am sure, but it is just in the starting phases, so it’s best to give it time and see where it goes.”
Donahue, who raves about the glasses, said he has been able to import and share photos. He can make calls and send text messages. On a night out for dinner with his wife, who forgot her reading glasses, he was able to “read” the menu at the restaurant. He also uses them to use a sewing machine to sew aprons for a government contract as part of VisAbility Pittsburgh’s industries division.
The glasses can also connect a person with Be My Eyes, a network of sighted volunteers who, through a live video call, offer visual assistance with tasks such as matching colors, checking if the lights are on or preparing dinner.
“Limitations shouldn’t be put on you because you can’t see,” said Leslie Montgomery, chief operating officer at VisAbility Pittsburgh.
“Every person we show the Meta AI glasses to says ‘we want these,’” Petach said. “So many times I hear someone say, ‘I lost my vision and I thought my life was over and then I found you guys.’”
More than technology
It is best not to rely solely on technology, said Lee Nasehi, president and CEO of St. Louis, MO-based VisionServe Alliance, a consortium of U.S. and Canadian organizations serving people of all ages with blindness and low vision.
“It is best to learn the skills without a device and then use the device to enhance the quality of life,” said Nasehi, who has a 44-year-old son who is blind, has cerebral palsy and cognitive disabilities. “We tend to accept what AI says. What AI does is synthesize what is out there in cyberspace and there are some misconceptions. Also, what happens if the internet is down?”
Nasehi admitted she is not a technologically astute person but is trying to integrate AI into her professional life and knows other professional member organizations are doing the same thing.
“That is great, because … everything we do is visually based,” Nasehi said.
AI suitcase
Imagine rolling a suitcase through the airport that alerts you to your gate, restaurant or baggage claim. The AI suitcase or Cabot Project (Carry on Robot) aims to build a guiding robot for people with visual impairments and was created by Chieko Asakawa, a blind computer scientist and IBM researcher in 2014.
The suitcase has a 360-degree LiDAR, which is used in many autonomous vehicles. It calculates distances between the user and objects, both stationary and mobile. A mobile phone app is used to program a destination into the suitcase, which plans a route and directs the user through vibrations in its handle. The suitcase also features facial recognition technology and can notify the user if a friend is nearby.
The concept is in collaboration with IBM, other Japanese companies, and Carnegie Mellon University, where Asakawa is an IBM distinguished service professor at the Robotics Institute. Asakawa’s hope is that the suitcase can help people travel independently.
A person holds the handle and it guides them away from obstacles and can describe what is going on in the environment. Currently, it is mapping in a space inside CMU and other locations such as Pittsburgh International Airport as well as two other locations in Japan, where it is collecting data about its surroundings.
”An airport is a huge open space and there are lots of things and people,” Asakawa said via email.
Daisuke Sato, a senior project scientist and engineer at Carnegie Mellon University, said it can detect if you are too close to something. Sato has been working for Accessibility Research Group in IBM Research since 2005 to help bring this technology to help in real-world situations.
“It acts as your eyes,” Sato said. “If you are at the airport, it can take you to shops and gates and has a navigation system to guide you. Artificial Intelligence can help people understand their world.”
The suitcase weighs 30 pounds and is filled with equipment. Currently, there isn’t room for clothing and accessories, but one day that could be possible, Sato said.
“The hope is for it to do its own navigation,” said Human-Computer Interaction Institute PhD student Howard Han. “It feels good to help people. Human and computer interaction in the real world provides a nice pipeline. We want to analyze and collect data and connect with people. Better information equals better outcomes.”
A study by the American Foundation for the Blind said in recent years, artificial intelligence systems have grown in capability and versatility. When using AI, it is important to be cautious, the study said.
“As the capabilities of AI and automated systems expand, there is much excitement about the potential for autonomous vehicles, AI-enabled tools at school and in the workplace, and other innovations that could increase human efficiency,” the study said. “Many of these new technologies can help promote inclusion and accessibility. However, like all new technologies, AI can also pose risks for people with disabilities.”
Experts agree that AI can sound confident even when making mistakes in image descriptions, the study said. Such false confidence can make it difficult for users who are blind or have low vision to judge the accuracy of visual descriptions.
For something like having it read the dosage and medication on a prescription bottle, the device is taking a photo of the words and then AI is reading the letters, Miller said.
“It is going to read what is there,” Miller said. “It might be more difficult to read the text on the bottle because it’s not flat or if the text is a smaller font size.”
Donahue realizes that AI is not perfect, but said the glasses have increased his independence.
“It is by far the largest advancement in tech I have seen,” Donahue said. “It helps me do so many things.”
Including finding his dog.
JoAnne Klimovich Harrop is a TribLive reporter covering the region's diverse culinary scene and unique homes. She writes features about interesting people. The Edward R. Murrow award-winning journalist began her career as a sports reporter. She has been with the Trib for 26 years and is the author of "A Daughter's Promise." She can be reached at jharrop@triblive.com.
Remove the ads from your TribLIVE reading experience but still support the journalists who create the content with TribLIVE Ad-Free.