Lucy Edwards., age 27, is blind in both eyes. She and her sighted fiance and their two guide dogs have built a comfortable life for themselves in Birmingham, England. However, there are times when she requires assistance that he is unable to provide, and these are typically “rudimentary questions” that pertain to performing routine tasks. It may be as simple as telling the difference between two types of canned beans so that she doesn’t accidentally mix baked beans with her chilli.
Be My Eyes has been connecting people with volunteers who can describe something in front of them or walk them through a process via video call to help blind and low-vision people like Edwards navigate these routines since 2012. However, some users feel awkward about asking for assistance and instead choose to muddle through on their own. Currently, a beta version of the app is being tested by a select group of users that includes support for Chat GPT-4. Users are impressed by the level of detail and speed, and this feature may prove to be one of the AI chatbot’s most useful applications in helping people achieve greater autonomy.
The original version of the voice-enabled Be My Eyes app allows users to take a photo and immediately call one of the nearly 6 million sighted volunteers in its database to decipher what they need from the image. This service is available 24/7, in 150 countries and 180 llanguages. They could aid in navigation, color recognition, and the identification of perishables in the fridge. “It’s this lovely merger of technology and human kindness,” CEO Mike Buckley says.
What Hans Jrgen Wiburg Discovered?
Motivated by a desire to lend a hand, volunteers register for programs. They get calls at all hours of the day, regardless of what time it is where they are. Hans Jrgen Wiburg, the founder of the company and a visually impaired Danish furniture craftsman, discovered that blind users aren’t making the most of the service. According to an internal survey, many employees would rather not have to constantly rely on other people for help, while others are uncomfortable calling a stranger to ask for personal advice.
Then along came Chat GPT-4, the latest iteration of an AI tool that provides instantaneous responses to a variety of human prompts, iincluding knowledge-based questions and creative writing tasks like song lyrics and cover letters.
GPT-4’s ability to read and understand photographs is also crucial to Be My Eyes. The Virtual Volunteer, an AI chatbot, was built into the app. Since February, a beta group of users (Buckley won’t say how many, but it’s less than a hundred) have been testing the version in its early stages.
Initial Be My Eyes testers were very impressed with the system’s speed, depth of analysis, and ability to recognize and describe visual scenes. To illustrate, you can show a picture of the contents of your refrigerator and get not only a list of what’s inside but also suggestions for meals you can make using those ingredients and even links to full recipes.
One of the beta testers is Edwards, a content creator and disability advocate (as well as an investor in Be My Eyes). She became a user of Be My Eyes after losing her sight at age 17 due to a rare genetic condition.
How Does Chat GPT Version Works?
Edwards is amazed by how quickly the Chat GPT version works. Recently, she snapped a photo of the Heathrow Underground station, and the app provided her with step-by-step directions to Central London within five seconds. It’s faster than any sighted person on the planet, she exclaims.
Chat GPT’s elimination of human opinion is an additional benefit over human help. Instead of guessing or debating with her partner or friends about the name of a particular shade of eyeshadow, she can simply show it a picture of her palette and it will tell her the exact, true colors.
The company has not yet announced when the GPT version will be made available to the public, but it will remain free. The primary focus is on user safety and promoting responsible behavior. It’s meant to supplement other digital and physical aids for the visually impaired. Buckley says, “This does not replace a white cane or a guide dog.” Don’t use this as a street crossing, okay?
The Virtual Volunteer is still in its developmental stages, so it makes mistakes occasionally. It can produce what are called “hallucinations” in the industry when it interprets an image incorrectly, such as mistaking a toaster for a slow cooker or a Sonos speaker for an air purifier. But, as Buckley points out, it’s always getting better and more refined. Also, Buckley is adamant that they will not eliminate the volunteer community, so the option of a human call is always presented.
To Summarize
Even so, Edwards has no desire to return to the first edition. Human contact is nice, but she has plenty of that already. She thinks the GPT version is a useful tool and looks forward to seeing how the technology develops. She believes that the more AI develops, the more independent she will become despite her blindness. I thought I had stopped doing that a decade ago. I was sure I’d never see that money again.
