Prototype, assistive technology
SceneScribe, a wearable assistant for people with visual impairments
A headset and a cane that answer a spoken question about what is in front of the wearer, built for people with visual impairments.
A hardware, B software, C AI
What is SceneScribe?
SceneScribe is a wearable assistant for people with visual impairments, built by WhiteHat Design Studio as a prototype. A description is only useful if it answers the question that was asked. Someone at a shelf wants to know which chocolates are there, not a paragraph about the shop. SceneScribe keeps its answers short by putting the wearer’s question first: the agents read the image through that question, and only what matters comes back.
The hardware is small: a Raspberry Pi, a camera and a microphone inside a headset shell. The cane carries its own ESP32 and connects to a Wi-Fi access point that the headset creates itself, so the two pieces find each other without any other equipment.
How SceneScribe works.
Ask
The wearer holds a button and asks a question, such as which chocolates are on the shelf in front of them.
Capture
The camera in the headset takes an image of what is in front of them.
Read the scene
The question becomes text. An explainer agent reads the image with the question, and a wellness agent works on its reply, so the answer keeps to what was asked.
Answer
The reply is spoken back through the connected earphones.
Sense the way
An ESP32 in the cane joins the headset's own Wi-Fi access point and handles obstacle detection.
On film.
Clips from the log. They have no sound, and each one says what happens.
Questions about SceneScribe
What is SceneScribe?
SceneScribe is a prototype wearable assistant for people with visual impairments, designed and built by WhiteHat Design Studio. A headset and a cane answer a spoken question about what is in front of the wearer.
How does SceneScribe answer a question?
The wearer holds a button and asks. The headset camera takes an image, the question becomes text, a chain of language model agents reads the image through the question, and the answer is spoken through the earphones.
Is SceneScribe a finished product?
No. SceneScribe is a prototype, and no user trial results are published. It describes what the camera sees and does not replace a guide dog or a person.
What we do not claim
- SceneScribe is a prototype. No user trial results are published.
- It describes what the camera sees. It does not replace a guide dog or a person.
Tell us what has to work.
One sentence is enough. An engineer will reply.