Case Study
Vocable logo

Because we all want to be heard.

It began with one of our own.

When former WillowTree designer Matt Kubota’s partner Ana was diagnosed with Guillain-Barré Syndrome, we knew we had to help.

The rare autoimmune disorder left Ana temporarily paralyzed and unable to speak. So Ana and her caregivers looked to augmented and alternative communication (AAC) devices. But as they discovered, these devices are expensive, bulky, and outdated. While Ana recovered from the disease, she could only painstakingly communicate by blinking as caregivers pointed to individual letters on a posterboard.

Matt knew he and the WillowTree team could create a better solution for AAC users like Ana: stroke and trauma survivors, people with MS, non-verbal individuals, intubated COVID-19 patients and many more.

That vision became the Vocable AAC app.

It started on a WillowTree-wide Innovation Day and eventually became Vocable — a free, mobile platform for users to communicate via head and eye movements, 
or text-to-speech.

Download on the Apple App Store
Get it on Google Play

Voice: a fundamental human need.

$15,000

approximate cost of hi-tech speech generating devices

17.9M

adults experienced difficulty speaking in the past year

1 in 12

children between ages 3–17 are affected by voice, speech, language, or swallowing disorders

1 in 3

stroke survivors have problems speaking, reading, or writing

We created Vocable because we believe in the power of voice.

We believe shared languages lie at the root of every ecosystem. Language is what connects every JavaScript programmer, every caregiver and child, and every doctor and patient, creating something greater than ourselves: community.

Tracking Eye Movements

And the power of the WillowTree community proved itself to Matt and the Vocable development team time and time again. In the product’s first iteration — “Eye Lasers” — users experienced significant shakiness when trying to control the cursors that tracked their eye movements.

Engineered for Stability and Precision

Surprisingly (or perhaps not surprisingly to Star Wars and robotics fans everywhere), WillowTree senior software engineer Chris Stroud’s love for building motorized droids held the solution. Using the codebase from Chris’s robot “child” BB-8 — a droid whose head balances on a movable ball — and PIDA controller design, the team gave users a more stable, precise experience when manipulating keyboards of any size and on any device.

Watch more of Chris’ BB-8 demo video.

“We’ve received positive feedback from the nurses working with COVID-19 and ventilator patients, including some heartbreaking stories of patients using the app on iPads to communicate with family members for the last time. [WillowTree’s] donated technology makes it possible for people in very difficult situations to communicate with caregivers and family members in a way they would not have been able to otherwise.”

Derick Rhodes
gofundme organizer — communication tools for ventilator patients

Obsessed with the micro to drive macro impact.

Vocable’s head-tracking custom API follows facial features and movements ranging from head nods to the most subtle eye twitches.

Personalized Pattern Detection

In doing so, users control a cursor that selects letters, custom phrases from personalized categories, and automated suggested responses. The app’s machine learning-powered, predictive pattern detection powers a deep semantic understanding of an individual user’s speech and offers a faster, smoother, and better experience for today’s AAC users.

Activating Listening Mode

Vocable is a good listener, too. Through Listening Mode, Vocable offers suggested responses to yes/no questions, either/or questions, and questions that can be answered with a number. Users or caregivers can quickly and easily activate the feature by saying, “Hey, Vocable.”

Equal parts function and form.

“It’s very customized, such as how we highlight buttons and the cursor’s focus as you navigate: others might put a rectangle around elements, but we animate and make them pleasant,” said Sashank Patel, WillowTree Engineering Director. “Part of our philosophy was that engaging with the app should be delightful.”

Research and Design Considerations

If users have to spend so much of their day interacting with the Vocable app, it’s gotta look good, too, right? Based on feedback from research partners like the assistive technology teams at Duke University and speech language pathologists, WillowTree’s design team identified ways to accomplish exactly that.

Accessibility and Delight

Guiding our design work was the principle that high contrast can be so much more than black and white. That’s why Vocable incorporates the strongest ADA compliance through elements like a compelling high-contrast palette and large typography, promising a consistent and aesthetically pleasing experience across devices.

Because what’s accessible should also be enjoyable.

“WillowTree believes voice is the future. Vocable is the most powerful voice product we could ever create... I’ve worked at WillowTree for over six years and this is by far one of the most impactful projects I’ve been on, by any means — and I worked in healthcare IT prior to WillowTree.”

Kylie Kalik
WillowTree Product Architect Director

We’re setting the tone for the future of voice.

Voice is powerful. It’s also deeply personal. We tapped into iOS native capabilities to offer text-to-voice generators that more closely match an individual’s voice.

Serving Users Around the World

Then, we took the local and made it global. With the help of the localization management platform Crowdin, Vocable became a multilingual product that could serve people speaking different languages around the world.

The Next Iteration

It’s open-source, too. So any engineer like Chris, designer like Matt, or product architect like Kylie can contribute to Vocable — anytime, anywhere.

Download the code here: iOS | Android

Vocable gives voice to WillowTree’s commitment to connect and represent all of us. And Vocable’s story isn’t over — not even close.

It’s got plenty more to say.

Connect with us to offer Vocable feedback or chat with our developers.

This website uses tracking technologies to analyze site usage and enhance the experience of the website. In addition, if you click “accept” we may also use tracking technologies that are controlled by third parties who are not our service providers. [i.e., Beeswax; Liveramp; Google Analytics]