When former WillowTree designer Matt Kubota’s partner Ana was diagnosed with Guillain-Barré Syndrome, we knew we had to help.
The rare autoimmune disorder left Ana temporarily paralyzed and unable to speak. Ana and her caregivers looked to augmentative and alternative communication (AAC) devices. But they soon discovered the available options were expensive, bulky, rudimentary, or outdated. While Ana recovered, she painstakingly communicated by blinking as caregivers pointed to letters on an alphabet poster.
Matt knew he and the WillowTree team could create a better experience for Ana and other AAC users: stroke and trauma survivors, people with MS or ALS, non-verbal individuals with autism, intubated hospital patients, and many more. That vision became the Vocable AAC app, a revolutionary augmentative and alternative communication tool, now AI-enabled with the integration of ChatGPT.
It started on a WillowTree-wide Innovation Day and eventually became Vocable — a free, conversational AI mobile platform allowing non-verbal users to communicate via subtle head movements and text-to-speech.
Winner | Experimental & Innovation Best Practices
Honoree | Accessible Technology
Nominee | Best Use of Voice Technology
Techquity Case Study Award 2024
$15,000
approximate cost of hi-tech speech generating devices
17.9M
U.S. adults experienced difficulty speaking in the past year
1 in 12
children between ages 3–17 are affected by voice, speech, language, or swallowing disorders
1 in 3
stroke survivors experience difficulty speaking, reading, or writing
Sources: U.S. figures from the National Institute on Deafness and Other Communication Disorders, Stroke Association, Your ALS Guide.
We believe shared language lies at the root of every relationship. Language connects every parent and child, every caregiver and patient, every programmer and piece of technology, creating something greater than ourselves: community.
“Vocable is making a major difference in the ALS community. I have seen people break into tears when using your app.”
Paralyzed users like Ana, who can’t tap a screen, needed an innovative way to manipulate an AAC device, and the power of the WillowTree community proved itself to Matt and the Vocable development team repeatedly. In the product’s first iteration — “Eye Lasers” — users experienced significant shakiness when trying to control the cursors that tracked their eye and head movements.
The robotics expertise of WillowTree senior software engineer Chris Stroud unlocked the solution, providing a more stable, precise experience when manipulating elements of any size and on any device. With subtle head movements, users could now control a cursor to type letters or select custom phrases from personalized categories.
The integration of OpenAI's ChatGPT allows the app to retain conversational context, enhance predictive pattern detection, and bolster semantic understanding of a caregiver’s speech, offering a faster, smoother, and more natural conversational experience for today’s AAC users. It's an ideal use case for conversational + generative AI.
Vocable is a good listener, too. Through Listening Mode, Vocable understands caregiver speech and context, offering users AI-generated responses that carry the conversation forward. Users or caregivers can quickly and easily activate Listening Mode by tapping the mic icon or saying, “Hey, Vocable.”
“ChatGPT is a conversational interface, it's literally made for this. It understands an entire conversation's history, providing better AI-generated responses for the non-verbal user to select."
If users have to spend so much of their day interacting with the Vocable app, it’s gotta look good, too, right? Based on feedback from research partners like the assistive technology teams at Duke University and speech-language pathologists, WillowTree’s design team identified ways to accomplish exactly that.
Guiding our design work was the principle that high contrast can be so much more than black and white. That’s why Vocable incorporates the strongest ADA compliance through elements like a compelling high-contrast palette and large typography, promising a consistent and aesthetically pleasing experience across devices. Because what’s accessible should also be enjoyable.
“When so few hospitals have available technology for the communication-vulnerable, it's exciting that a platform like Vocable exists. Vocable is easy to access and most importantly free."
Vocable is global. With the help of the localization management platform Crowdin, Vocable became a multilingual product that can serve people speaking more than a dozen different languages around the world.
“WillowTree believes voice is the future. Vocable is the most powerful voice product we could ever create... I’ve worked at WillowTree for more than seven years and this is by far one of the most impactful projects I’ve been on, by any means — and I worked in healthcare IT prior to WillowTree.”
Interested in focusing your conversational AI initiatives?
Let's talk
