The McGurk Effect: Why You Should Look When People Speak

July 21st, 2014 by Melinda 'Sunni' McBride, Au.D. Comments »

visual cues and hearing comprehension

Even with hearing aids, living with hearing impairment can be challenging. That’s why it’s important to understand how you can make the most of your hearing ability. One of the easiest ways to do this is simply by watching a person’s lips when they are speaking.

The McGurk effect

Also known as the McGurk-MacDonald effect, this perceptual phenomenon occurs when a spoken syllable is paired with a different but similar lip movement. A listener will hear one syllable with their eyes closed, and another syllable as they watch a speaker’s lips.

In most classic McGurk experiments, the syllable in question is “ba.” When “ba” is visually synced with the lip movement of a different syllable, “da”, listeners will hear “ba” with their eyes closed and “da” when they watch the speaker’s lips. In fact, the change in sound is so strong that a listener will continue to hear “da” even after they know the speaker is actually saying “ba.”

If you want to see how this works (or if you aren’t buying it!), take a look at this video,

While the McGurk effect is a perceptual “trick” of sorts, it effectively demonstrates the relationship between hearing and seeing. Visual cues will affect what we hear, and can easily make us second guess auditory information. According to a recent 2013 study conducted at the University of Utah, the overriding effect of visual cues can’t be turned off, either.

“We found vision is influencing the hearing part of the brain to change your perception of reality — and you can’t turn off the illusion,” says Elliot Smith, a bioengineering author of the study. “We’ve shown neural signals in the brain that should be driven by sound are being overridden by visual cues that say, ‘Hear this!’”

What the McGurk effect means for the hearing impaired

For the hearing-impaired to take full advantage of the McGurk effect and hear more precisely, they should always look at a speaker’s lips. By doing so, a listener is effectively bolstering their hearing ability with visual input, which can be much more reliable.

Conversely, the hearing-impaired should not look away from a speaker during conversation — they could end up hearing something else entirely! (Which, as I’m sure we all know, happens fairly often.) Looking at a speaker and “seeing what they’re saying” is a simple but incredibly efficient listening technique that can greatly improve quality of life and allow hearing-impaired individuals to step smoothly into conversations with confidence.