The McGurk Effect

The McGurk effect (named after Harry McGurk of McGurk & McDonald, 1976) is a compelling demonstration of how we all use visual speech information. The effect shows that we can't help but integrate visual speech into what we 'hear'. Try the demonstration now, and then read about how the stimuli were made, what the effect means, and how to produce your own McGurk effects.

Instructions: You will see and hear a mouth speaking four syllables. Watch the mouth closely, but concentrate on what you're hearing. The movie will repeat itself until it is stopped.You can watch it as many times as you need to be sure of the syllables you hear. After you feel certain of what you perceive, stop the movie and continue reading the text below.
PLEASE START THE MOVIE NOW.

Now start the movie again and close your eyes. Listen to the movie repeat until you are sure of what you hear. When you feel certain of what you hear, stop the movie and continue reading the text below.
PLEASE RESTART THE MOVIE NOW.

If you're like most people, what you hear depends on whether your eyes are opened or closed. If you'd like, you can start the movie again, and as it repeats, switch between opening and closing your eyes. Your experience of what you hear should change. After you're convinced, stop the movie and read the explanation below. You can always play the movie again later.

How the stimuli were made: These stimuli were made by dubbing a single repeated audio syllable onto four different visual syllables. To see what syllables were used for this demonstration, please point your cursor below.

What the effect means: The McGurk effect shows that visual articulatory information is integrated into our perception of speech automatically and unconsciously. The syllable that we perceive depends on the strength of the auditory and visual information, and whether some compromise can be achieved. Regardless, integration of the discrepant audiovisual speech syllables is effortless and mandatory. Our speech function makes use of all types of relevant information, regardless of the modality. In fact, there is some evidence that the brain treats visual speech information as if it is auditory speech .

How general is the McGurk effect?
The effect works on perceivers with all language backgrounds (e.g., Massaro, Cohen, Gesi, Heredia, & Tsuzaki, 1993; Sekiyama. & Tokhura, 1993)
The effect works on young infants (Rosenblum, Schmuckler, & Johnson, 1997).
The effect works when the visual and auditory components are from speakers of different genders (Green, Kuhl, Meltzoff, & Stevens, 1991).
The effect works with highly reduced face images (Rosenblum & Saldaña, 1996).
The effect works when observers are unaware that they are looking at a face (Rosenblum & Saldaña, 1996).
The effect works when observers touch—rather than look—at the face (Fowler & Dekle, 1991).
The effect works less well with vowels than consonants (Summerfield & McGrath, 1984).
The effect works less well with nonspeech pluck & bow stimuli (Saldaña & Rosenblum, 1994).
The effect works better with some consonant combinations than others (e.g, McGurk & MacDonald).

To produce a 'live' demonstration of the McGurk effect: (you'll need two other people besides yourself)
1) have an observer face you and keep looking at your face
2) have another person stand behind you so the observer can't see their face
3) starting synchronously, repeatedly mouth the word 'vase' (silently) while the person behind you repeats the word 'base' outloud -you can acheive synchronization by counting down '3, 2, 1. . vase, vase, vase, etc
4) after about 8 repetitions, stop and ask the observer what they 'hear' -they should 'hear' vase
5) now do the same thing, and this time tell the observer to shut their eyes after a few repetitions
6) they should hear 'base' with their eyes shut
7) the observer can try opening and shutting their eyes, and what they 'hear' should change from 'vase' to 'base'

Some tips on making your own McGurk Stimuli: Audiovisual dubbing can be achieved by using two videotape players or digitizing stimuli onto a computer and using software to mix the audio and video components. The qualtiy of the auditory channel should be good, but the quality of the visual channel can be fair without much loss in the effect. The auditory and visual components should be synchronized so that the sound of the syllable seems to be coming from the visible mouth. However, the components do not have to be perfectly synchronized for the effect to work. The syllable combinations used in the above demonstration are known to be especially strong.To see a picture of equipment used to create McGurk stimuli, CLICK HERE.