TY - JOUR
T1 - The impact of cross-modal correspondences on working memory performance
AU - Brunetti, Riccardo
AU - Indraccolo, Allegra
AU - Mastroberardino, Serena
AU - Spence, Charles
AU - Santangelo, Valerio
N1 - Publisher Copyright:
© 2017 American Psychological Association.
PY - 2017/4/1
Y1 - 2017/4/1
N2 - Cross-modal correspondences influence perceptual performance in adults, infants, and even nonhuman primates across a variety of different sensory modalities, including tasks involving speeded detection and categorization. However, to date, it is still unclear whether and how correspondences could modulate post-perceptual processes, such as working memory (WM). We investigated this issue using an audiovisual two-back task. In Experiment 1, 3 kinds of correspondences were used: audio/visual numerosity, pitch/shape, and pitch/elevation, each presented congruently (e.g., for numerosity: 3 tones along with 3 shapes) or incongruently (3 tones/2 shapes). Participants attended to the visual or auditory modalities, or both, simultaneously. The results revealed faster target-detection latencies following congruent as compared to incongruent stimulation, especially for numerosity congruence. In Experiment 2, we focused on numerosity, varying the correspondence of the unattended modality, thus having correspondences at both sample (e.g., 3 tones/3 shapes) and target (e.g., 3 tones/3 shapes), only at sample (sample: 3 tones/3 shapes; target: 3 tones/2 shapes), only at target (sample: 3 tones/2 shapes; target: 3 tones/3 shapes), or never. To investigate the information format we included "symbolic" quantities (i.e., visually/auditorily presented digits). The results confirmed the congruence effects, specifically when the correspondence operates at the target display, thus affecting response selection. The experiment revealed modal effects, showing how task-irrelevant digits affect performance only in the auditory modality, while task-irrelevant quantities affect it only when presented visually. Overall, these findings highlight the impact of cross-modal correspondences on WM, adding new light on the link between perceptual and postperceptual stages of human information processing.
AB - Cross-modal correspondences influence perceptual performance in adults, infants, and even nonhuman primates across a variety of different sensory modalities, including tasks involving speeded detection and categorization. However, to date, it is still unclear whether and how correspondences could modulate post-perceptual processes, such as working memory (WM). We investigated this issue using an audiovisual two-back task. In Experiment 1, 3 kinds of correspondences were used: audio/visual numerosity, pitch/shape, and pitch/elevation, each presented congruently (e.g., for numerosity: 3 tones along with 3 shapes) or incongruently (3 tones/2 shapes). Participants attended to the visual or auditory modalities, or both, simultaneously. The results revealed faster target-detection latencies following congruent as compared to incongruent stimulation, especially for numerosity congruence. In Experiment 2, we focused on numerosity, varying the correspondence of the unattended modality, thus having correspondences at both sample (e.g., 3 tones/3 shapes) and target (e.g., 3 tones/3 shapes), only at sample (sample: 3 tones/3 shapes; target: 3 tones/2 shapes), only at target (sample: 3 tones/2 shapes; target: 3 tones/3 shapes), or never. To investigate the information format we included "symbolic" quantities (i.e., visually/auditorily presented digits). The results confirmed the congruence effects, specifically when the correspondence operates at the target display, thus affecting response selection. The experiment revealed modal effects, showing how task-irrelevant digits affect performance only in the auditory modality, while task-irrelevant quantities affect it only when presented visually. Overall, these findings highlight the impact of cross-modal correspondences on WM, adding new light on the link between perceptual and postperceptual stages of human information processing.
KW - Bimodal n-back
KW - Cross-modal binding
KW - Cross-modal correspondences
KW - Modal/amodal representations
KW - Multisensory integration
UR - http://www.scopus.com/inward/record.url?scp=85016242920&partnerID=8YFLogxK
U2 - 10.1037/xhp0000348
DO - 10.1037/xhp0000348
M3 - Artículo
C2 - 28345948
AN - SCOPUS:85016242920
SN - 0096-1523
VL - 43
SP - 819
EP - 831
JO - Journal of Experimental Psychology: Human Perception and Performance
JF - Journal of Experimental Psychology: Human Perception and Performance
IS - 4
ER -