Our Inclusive Design 24 (ID24) presentation

Voice UX is an emerging mode of interface navigation but for users with speech and language difficulties and in particular stuttering, this presents a new way that barriers can be created.

But first some context about the condition and my own experience and this isn’t necessarily reflective of the entire community of people who stutter. Stutter or stammer basically the terms are the same and it describes dysfluency in the pattern of speech.

It's where the speech can be categorised by repetition of words or sounds, blocking and an inability to get words out, excessive use of filler words, rapid speech or using physical ticks like tapping a finger and tapping a hand to get words out.

It affects 1% of the worlds population which is roughly 70 to 80 million people. Most grow out of it, but boys are 3 – 4 times more likely than girls to continue stuttering into school age and adulthood.

It can be roughly categorized as overt and covert. An overt stutter is one that is visible on the person, many words, sounds and situations are challenging, and you notice they're having difficulty getting words out. Whereas covert is where the stutter is well hidden. The person may replace words, avoid situations, or use strategies to avoid talking all to mask and hide the stutter.

Whilst there are many therapies and programmes available that attempt to reduce the effects of stuttering, there is no cure. More broadly stuttering isn’t always seen as a disability. Public perception and people sometimes assume the person is lacking confidence, or they're tongue tied, or as I've experienced casual insensitivity where people have said "slow down you're talking too fast" not necessarily understanding that stuttering is a disability and talking fast is my way of managing it.

But perhaps most ironic of all is the film unspoken truth submitted by Dutch film maker Irmo Stijnberg (ear-moh steyn-beark) it was disqualified from the European film festival Integration you and me (a festival devoted to disability) in 2025 because the festival claimed stuttering isn't a disability,

I've had a stutter all my life, my mother had a stutter as did her father (my grandfather) and there appears to be some hereditary link. Theirs vanished overnight when they left school and entered adult life. My mum often said, it was practically overnight that hers vanished. I too assumed once I left school mine too would vanished, but it persisted.

I thought on leaving college it would vanish but no it was still there, finally graduating university meant it would surely vanish but no its still there. Short of taking on further education my stutter has persistently stuck with me.

My stutter is categorised by repetition of words and long pauses. I know what I want to say but sometimes it takes time to formulate the words. I've used strategies in the past of talking at a fast speed to maintain the flow of words because for me when I stop that's when it becomes difficult to start again.

Other strategies have been changing and switching difficult words and sounds for easier words and sounds. I become so good at switching words that I inadvertently create a list of difficult words which reinforce subconsciously that I can never say that word again, and it makes what I'm saying more difficult to understand.

Stuttering generally affects words which begin with a consonant, a hard letter. So letters like C, D, P, R, K, M, T for example the words Drive because of the letters D and R and Two because of the letter T are challenging, whereas vowels and words beginning with them are generally easier to say.

I've attended speech therapies and programmes which have helped to a degree. For many years I attended the McGuire programme. This is a global residential programme which aims to make the act of speech easier. But after 10 years of attendance, I found the techniques wore off and it became challenging to talk without dysfluency outside of the comfort zone of the programme.

The programmes techniques focus on deeper breathing from the costal diaphragm, a little bit like how opera singers perform. The theory is a deep breath gives fuel to the words overcoming the tendency to block. Other techniques include speaking with a small number of words per breath.

The programme discuses stuttering as much more psychological than a physical impediment. What I mean is the minor dysfluency people encounter is magnified by their own thoughts which makes the stutter worse. It's self-fulfilling, I stutter, therefor I will stutter in all situations of great tension or anxiety.

For all the programmes and therapies I've attended, for me I've found my stutter stubbornly remains and has become a pattern of habit-forming speech.

Due to the condition, I've been frequently made looked foolish where I've blocked and paused when asked my name, or my age, or my address. Knowing full well all those details, only for a person to reply "don’t you know your name" … I can talk for all stutterers with this experience, that is particularly frustrating.

So that's the background and my experiences, but what can technology do to help make the condition easier for people who stutter. I separate this presentation into 2 themes. The first being voice interfaces that adapt to users' speech by following a set of design principles and the second assistive technology as a learning aid to help the user whilst speaking.

The W3C Cognitive and Learning Disabilities Accessibility Task Force have published a draft cognitive accessibility research module – voice systems and conversational interfaces.

This already has some really good recommendations and guidance. Such as correctly Interpreting Spoken Information, issues responding to Voice and Speech Recognition Systems, allowing users to change settings by using natural language and to follow the standards and best practices in the Association for Conversational Interaction Design wiki and European Telecommunications Standards Institute technical report.

These are all good first steps, but I think we can go further. I've developed a set of design principles for voice interfaces which describe behaviour which can be adopted.

Principle 1. Provide adaptive pause tolerance

Stuttering is not just the repetition of words or sounds, it's the hesitancy before words, hesitancy between sentences and even within individual words, it's the pausing and the breaking up of individual words and this is all entirely situational.

For me, I can be fluent when talking in private or with close family members. Yet when talking in public, in unfamiliar surroundings or high-pressure situations that fluency is reduced, the higher pressure the less fluent, and my speech becomes choppier.

Because of this, one statement can have a completely different meaning due to the placement of a pause, for example the phrase "I want to go to the hospital to see a family member who works there" … and I've had it happen many times, I'm halfway through a sentence and an errant pause causes the person to rush and complete the sentence (often incorrectly) for me. For everyday in person situations, it's not much of an issue. You finish the sentence correctly, it's a little awkward but otherwise fine.

But if a pause in speech occurs in a voice interface after "I want to go to the hospital", it may assume an elevated level of response is needed, perhaps it's an emergency, perhaps it's something important … yet it's just for visiting a relative and not a personal emergency. It's interrupted me before I've had time to finish.

The pause length of understanding when a sentence has finished needs to be adjustable. It needs to understand the potential for extended gaps between sentences, phrases and even individual words. Without an adequate pause time it'll begin assuming the word, sentence or phrase has finished and potentially reach an outcome which is incorrect.

You might think an obvious solution is to introduce a tolerance and have it set for extended pauses to match the person's speech pattern and this should solve the problem. But pausing is symptomatic of the situation. Talking in public with lots of noise, distraction and people waiting on you can increase the likelihood of the anxiety and can make the stutter worse.

Yet in a comfortable setting, the anxiety is low and speech is unaffected.

Because of this, the adjustment must be adaptive and situational, it isn't sufficient to have one setting for all. Talking in private may require a minimal level of tolerance versus talking in public may require a greater tolerance, the situations are different. This is adjusting the time spent waiting for a device to finish.

But it also raises the larger question just who decides how the Voice UI system adapts? Automatically detecting that someone is struggling and altering behaviour sounds helpful, but it could also feel invasive or unpredictable.

The ideal might be an adaptive system with clear user overrides rather than a system silently deciding, “The person is having difficulty speaking, so I'll change modes".

Principle 2. Allow recovery without repeating

The user may have just struggled for 20 seconds to say something and then be asked to start over. Can the person repair the interaction without repeating themselves? Can they correct one word, select from interpretations, edit the transcript, go back, or confirm what the system thinks it heard?

If all this requires additional voice input which also has poor error recovery, the person can begin spending a disproportionate amount of time correcting the systems assumptions or contending with multiple "I'm sorry I didn’t understand, please say that again … slower … louder … smoother".

Can the interaction be repaired without the person having to repeat themselves? As shown in the hospital example, early interpretation can result in the system taking a completely different path.

If voice recognition caused the problem, requiring another voice command to correct it can create a cascading loop.

Principle 3. Allow easy switching between modes

A person might start an interaction speaking, switch to typing when they hit a difficult word, select an option visually, then continue speaking all without losing progress. That is potentially even more important than perfect speech recognition. The accessible voice interface may actually be one that makes changing modality effortless.

But the progress must be retained between those modes, it isn’t sufficient to think the person is speaking therefor all interactions will be voice based.

Principle 4. Customisable wake commands

The way voice interfaces are activated or woken can be problematic. Can the wake prompt be adjusted or changed to a custom word or phrase? If it’s a fixed prompt which begins with a difficult word or sound, the struggle to activate the device increases.

You'll remember where I said consonants like the letters D, G, P, R can be difficult to formulate. If a device name begins with one of those letters you can see how difficult it can be difficult to trigger.

For example, the Google nest device, it's woken by saying "Hey Google", the G in Google is a hard letter, and potentially the user blocks on the word "Google" and begins repeating "Hey" with a pause and the device never wakes up as its never hearing its activation phrase in a complete sentence.

Cleverly Apple has pre-empted this and within the accessibility settings for iOS devices is the option to change the voice prompt from "Siri" or "Hey Siri" to something completely unique. Within the settings is the menu vocal shortcuts which you can change to any word to trigger Siri to an easier to say word.

I've tested it and whilst it works, it appeared to take longer than the regular voice activation prompt.

However, this custom activation approach doesn’t appear to work on Google products. Only variations of "Hey Google" or "Ok Google" are supported. And this is the same with Alexa devices. Alexa only appears to have one of 5 pre-selected wake words.

Principle 5. Provide adaptive dysfluency removal

Dictation is another challenge. Filler words are prevalent as a sentence is being constructed, words such as "um", "ah", "eh" are used to bridge difficult words or make the lead in to difficult words easier.

If the voice interface cannot adjust its tolerances for repetition of filler words, it begins to dictate the output as is. In a formal setting where clarity of thought and intent is important such as legal documents, or research papers these filler words aren’t helpful.

As part of their research at Apple Machine Learning researchers studied dysfluency refinement, basically programmatically removing these filler words and detecting unwanted repetitions.

Deleting unwanted repetitions though isn’t straightforward you can't simply have an algorithm to remove all occurrences of repetition because the English language contains repetition.

They give the example phrase "we had had many discussions" removing the duplicate changes the intended meaning of the sentence to "we had many discussions". Interestingly they found filler words would often be transcribed as "oh" and so it became a straightforward process to remove occurrences of "oh" unless it's representing a number.

For detecting unwanted repetitions, they used a language model to estimate how likely the repeated sequence occurs naturally, for example the sentence "I I I I I would like a coffee" has the language model identify the repeating I's as being very unlikely to occur naturally and so the sentence becomes "I would like a coffee". They use a probability model to determine how plausible those words belong next to each other.

They discuss that speech recognition and dysfluency refinement have different jobs, one is to accurately determine speech the other is recognise the person's intent removing words not related to this.

The results showed the approach works very well in intentional repetition but considerably harder for free-form conversation.

But again, we come back to this decision, is it up to the system to decide to remove dysfluency. And dysfluency shouldn't automatically be treated as noise that technology should remove. The user should ultimately decide whether technology is interpreting, transcribing or refining their speech.

Principle 6. Recognise intent, not exact words

For saying specific words this can prove difficult. I briefly discussed it earlier with the wake-up commands for some devices being fixed. But more broadly if the voice interface requires a user to speak specific words in a specific sequence this too can become a barrier. For many stutterers, saying their name or their address is difficult if it begins with a hard consonant fortunately for my name "Ross" I don’t have that challenge but in other situations I do.

If a voice interface relies on spoken commands in a specific sequence this isn't good. Difficult words in everyday speech might get replaced or spoken in slightly different ways by the person, therefore the intent of the speech matters far more than the spoken words.

Take for example the phrase "the weather is really good today" the G in the word "good" can be difficult to say as can the T in "today", so someone might replace it with the phrase "the weather is really nice now" ….. both phrases mean the same but one is significantly easier to say due to the adoption of words which begin with softer letters.

But the intent of the message is exactly the same, both describe the weather as it is now.

The intent of the message is far more important than the actual words, try to avoid the voice interface requesting fixed words in a fixed sequence. Be flexible.

But where there is no avoiding specific words, such as names or addresses or even age. Understand that the word may come with additional textual baggage at the beginning. It might be a lead in word or phrase such as "my name is …." Or "my age is…" or even filler words, of "umms" or "ehhs". Often the first several words spoken, or even the first sound aren’t the requested information, but they're a part of the user's pattern of speech leading to the word.

Principle 7. Show what’s coming next

Off the cuff talking or adlibbing I find particularly challenging. You might be thinking my speech is fluent and there is no sign of dysfluency but I'm reading all my notes from a script, I've written it as I speak. If I were to try and speak in a more fluid and looser way, the formulation of words becomes difficult and my speech reverts back to being choppy, staccato or contain more pausing when I'm trying to think on my feet whilst talking.

If the user is asked to read prepared words or even recite a phrase consider showing more of the content to be read as well as the current words. I call this technique ghosting, where the further parts of the sentence are visible. This helps the user to mentally prepare or understand what is coming next. I'm unsure why exactly this works for me, but I've always found reading off a prepared script is far easier than without.

Spotify in fact do this very well with their podcasts. As you're listening, the app highlights the word spoken and shows the upcoming words. It isn't part of any speech interface, but it does provide a technique to show the upcoming content to be spoken.

On screen is a screenshot from the Spotify app, the Rest is History podcast is playing (if you've not subscribed I'd highly recommend it, it's great). The past spoken words of "so these are very" are in white, the current word "prescient" is highlighted in purple with white text, whilst the upcoming words "words from Major General Ford anyway" are greyed out but still visible.

If you're adding a speech interface to your digital product, it's unlikely you'll be creating your own and have total control over how it works and its internal logic. Instead, an existing speech engine will probably be used, and this is the challenge with existing products. If those speech engines don’t have tolerances, or can't adjust how they behave the barriers for interacting with it remain.

But that isn’t to say the technology giants aren't researching speech dysfluencies. Project Euphonia is a Google research initiative working to make speech recognition software more accessible. They’ve teamed up with the University of Illinois Urbana-Champaign for their speech accessibility project which has cross-industry support from Apple, Google, Meta, Microsoft as well as many nonprofits. The research isn’t only centred on stuttering but a diverse range of speech dysfluencies including Parkinson's disease, cerebral palsy, down syndrome and hearing loss.

They're recording samples of speech to more effectively train speech recognition technologies on these diverse datasets. Whilst a great initiative, with stuttering the condition is very situational. But I question would a person with a stutter record the speech prompt training data without any dysfluency in the comfortable surroundings of their home? Some might but many won't, so already you can see the dataset is being biased by only those with very prominent overt dysfluency.

This is why I think diverse datasets alone are only part of the solution. Also, this project is US based biased towards English speakers, and with stuttering, only those with severe overt stutters would be able to record a pattern of dysfluent speech for the technology to learn from.

I believe datasets and design principles addressing the mechanics of the speech interface are the way to go, ensuring the recognition technology supports users where the dysfluency may be situational.

But those are the principles I think can help guide better design decisions for voice-based interfaces but can technology also be used as a training aid?

Learning aids to establish a rhythm

With my speech, it's almost as if I've lost it's natural rhythm. When I talk it's matched with my breath and is usually several words bunched up together followed by a breath, that's how I've been able to reintroduce a rhythm, forcing myself to speak in that pattern.

In previous speech programmes, having a person visibly count me in to take a pause or counting a specific number of words does help. When I've had someone reminding me to pause after say 6 words, take a breath and start again it has been able to introduce more regular speech with less dysfluency.

And there are already apps available that are training aids for stutterers. Keep it costal is one such app where it attempts to train the person to costal breath.

Anecdotal research has shown introducing a metronome for a person to match their speech can also help. Each tick of the metronome corresponds with saying a word and it becomes formulaic where speech is having an artificial rhythm reintroduced.

From my own experience I know having a constant reminder of rhythm does help, previously I used my own finger to tap to establish a rhythm to my speech.

Technology can help with this. Many smart watches and devices have haptic feedback that use vibration to inform the user of alerts.

And it's this haptic feedback that can be used to help improve the rhythm of speech. A variable beat, expressed as a series of vibrations that reminds the user to synch their pattern of speech to an established rhythm. Take the phrase The …. Quick …. Brown …. Fox …. Jumps …. Over …. The …. Lazy ….. dog .., where after each word the device vibrates. It helps establish that rhythm which is sometimes lacking. If the person misses their mark, it's only a few moments before a new mark appears.

Now this tackles the rhythm of speech, introducing an artificial mechanism to help. But speech is so much more than just a rhythm it’s the taking of breath between the words.

For me, this has been categorised as talking on an ever-decreasing volume of air in my lungs, until I reach a point of blocking or speaking in a very strangled voice trying to wrangle the last of the oxygen.

Just as the people in the McGuire programme would face me, and remind me to breath after 5 words. That too can be implemented in device haptics, a regular beat for the rhythm of speech followed by a rapid beat informing the user to take a breath.

For example, again the phrase "the quick brown fox jumps over the lazy dog" as the phrase ends I imagine the device rapidly vibrates to physically signal the person to breathe. Now it sounds unusual to have a signal to breathe but with stuttering, the sensation to get all your words out, on a single breath, is very powerful to think about when you're in a stressful situation.

This rapid vibration would be adjustable, the sweet spot for talking a number of words before taking a breath I've found to be 5-8 words.

But the voice interface would need to actively listen to the users speech to understand how many words have been spoken and when to remind the user to pause. And this doesn’t even touch upon the whole range of privacy and ethical concerns having a device constantly listening and monitoring you.

But a system that helps train users to get words out in such a way I believe would be a help for some people with stutters.

How is success measure?

But how would success of a speech interface be measured? I think it should be measured by person effort, not just recognition accuracy. A system could achieve excellent technical speech-recognition accuracy and still be inaccessible because the person needed four attempts, changed three words, rushed their speech to beat a timeout and experienced considerable anxiety doing it.

An accessibility metric could therefor be interaction burden. The repetitions required, the number of interruptions, corrections, abandoned tasks, time pressure and whether the person had to change how they naturally communicate to make the technology work.

On screen I have a mocked-up example of a metric dashboard with the burdens broken down by their type including repetition, interruption, correction and abandoned task. And also showing the burden over time in a line chart showing it peaked just under the 1minute mark.

Conclusion

Whilst these principles are centred around stuttering, the principles are for all speech disorders. The key takeaway is diverse speech datasets support some users but to support more provide choice and adaptability to the speech interface for all users with all levels of dysfluency.

Fluency is an unstated interface requirement. A conventional speech interface can implicitly require the user to begin quickly, maintain a particular rhythm, use expected vocabulary, avoid long silences, respond within a fixed window and recover rapidly after an error. As long as this implied behaviour remains unchallenged, voice interfaces will be an added burden to people with speech dysfluencies.

I've a few links which go into some more detail about what I've discussed. Our make voice UX accessible information page, it lists the 7 principles discussed and can act as a prompt when designing accessible voice interfaces and includes a downloadable PDF.

Google's project euphonia page and the page from University of Illinois Urbana-Champaign for their speech accessibility project.

And an opinion piece YouTube video from the New York times on the challenges of suttering, and Links for UK charity stamma on how not everyone benefits from automated voice recognition. There's plenty of great background reading for supporting people who may have a stutter.

All of which can be found at canaxess.com.au/ID24 that's c a n a x e s s .com.au/id24 Thanks for listening and if you're interested in working tougher on your accessibility projects, reach out to hello@canaxess.com.au.