How vision affects sound processing

There are a pair of papers in the most recent issue of Current Biology that got my attention thismorning.

There’s an original research article (Kayser, Logothetis, & Panzeri, 2010), and a handy review (Ghazanfar & Lemus, 2010). Full refs below.

There has been a flurry of recent research indicating that visual stimuli can influence activity in what were previously thought to be purely “auditory” areas of the brain – from the (relatively) high levels of the auditory cortex (Kayser, Petkov, & Logothetis, 2008) right down to the brainstem (Musacchia, Sams, Skoe, & Kraus, 2007). Interestingly, the effects of visual stimuli on activity in the brainstem seem to be even larger in musicians, but that’s a story for another time. It’s been mostly assumed, naturally enough, that the enhancements in behaviour (in the form of decreased reaction times, detection thresholds etc) are due to enhanced neural responses in the brain elicited by multisensory stimuli. Until recently, this assumption has never really been tested. Kayser et al (2010) tested this, by comparing the amount of stimulus-relevant information carried by neural responses in the auditory cortex of alert monkeys, who were watching videos of monkey vocalisations.

In addition to finding enhanced activity to multisensory stimuli, they also found an interesting relationship with the auditory-only responses. As is often found in these types of studies, the amount of enhancement gained from the multisensory stimuli increased as the auditory-only responses decreased. That is, the biggest gains occurred when the auditory signal alone was weakest. Strong auditory responses were actually suppressed by the addition of visual stimuli. At the same time, the information content of the auditory cortex activity increased. So, when congruent auditory-visual stimuli are presented, the effect of information from the visual system seems to selectively suppress strong auditory activity, which in turn reduces its variability, and increases its information content. Conversely, weak auditory signals are amplified by visual information, but at the expense of increased variability and lower information content.

Interesting stuff!

Musacchia, G., Sams, M., Skoe, E., & Kraus, N. (2007). Musicians have enhanced subcortical auditory and audiovisual processing of speech and music. Proceedings of the National Academy of Sciences, 104(40), 15894-15898. doi: 10.1073/pnas.0701498104

Ghazanfar, A. A., & Lemus, L. (2010). Multisensory Integration: Vision Boosts Information through Suppression in Auditory Cortex. Current Biology, 20(1), R22-R23. doi: 10.1016/j.cub.2009.11.046
Kayser, C., Logothetis, N. K., & Panzeri, S. (2010). Visual Enhancement of the Information Representation in Auditory Cortex. Current Biology, 20(1), 19-24. doi: 10.1016/j.cub.2009.10.068
Kayser, C., Petkov, C., & Logothetis, N. (2008). Visual Modulation of Neurons in Auditory Cortex. Cereb Cortex. doi: 10.1093/cercor/bhm187

This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s