Mario Sánchez García, Erasmus Student, Dept. of CSIS, University of Limerick (Ireland)
Submission of the assignment "Multimodal Bounce" for the Interactive Multimedia module (CS4358). The objective of this assignment is to replicate the experiments from Watanabe's paper (Watanabe et al. (2001)) in order to document how including an own new factor could affect the results, and how is this factor related to the cross-modal perception.
We call a perception cross-modal when it involves the interaction between two or more different sensory modalities, such as sound and light. In Watanabe et al. (2001), the experiments are based in introduce a visual ambiguity, which solution is influenced by audiovisual interaction. A two-dimensional display is presented to the observers, where two visually identical targets are moving across each other so they can be perceived either to bounce off or to stream through each other.
In the article is explained how despite the ambiguous nature of the visual stimulus, observers usually show a strong bias to see how the targets stream through each other. However, various factors have been reported to increase the relative frequency to percept how the targets bounce. The most remarkable one is to play a brief sound in the moment the targets coincide in the same point (simultaneous sound), suggesting that cross-modal perception is involved.
Using this last factor, Watanabe et al. prove that:
- The bounce-inducing effect is reduced when the simultaneous sound is preceded and followed by other identical sounds.
- The attenuation of the bounce-inducing effect depends on auditory context and the saliency of the simultaneous sound.
- The auditory context does not alter perceptual properties of the simultaneous sound itself.
- Watanabe, K. and Shimojo, S. (2001). When Sound Affects Vision: Effects of Auditory Grouping on Visual Motion Perception. Psychological Science, 12(2), pp.109-116.