Abstract
<p>Multimodal learning is widely assumed to enhance learning outcomes by engaging multiple sensory and motor systems. However, empirical studies have primarily examined combinations of two modalities, leaving richer learning protocols largely unexplored. This gap is particularly relevant for real-life situations, where several systems typically interact. Literacy acquisition is a prime example, as it requires the integration of visual and auditory input as well as motor representations of letter and speech sound production. To explore the benefits and limits of complex multisensory-motor learning, we investigated how increasing multisensory-motor engagement beyond two systems affects the learning of symbol-sound associations. Specifically, adult participants learned novel pseudoletter-syllable associations under three conditions: bimodal (audio–visual), trimodal (audio–visual–writing), and quadrimodal (audio–visual–writing–speaking). Learning outcomes were assessed using tasks measuring knowledge of pseudoletter–syllable associations, visual perception of pseudoletters, and motor productions of pseudoletters through writing and naming tasks. Contrary to the general predictions of multimodal enrichment theories, performance decreased as the number of modalities increased, consistent with cognitive load accounts suggesting that additional processing demands can impair learning. To further test whether these costs resulted from insufficient processing time, the integration time-window was extended in the second experiment, with the different sensory-motor systems engaged sequentially rather than simultaneously. Under these conditions, the detrimental effect of additional modalities was still present, but the audio–visual–writing training selectively improved pseudoletter writing performance. Together, these findings challenge the common assumption that adding modalities necessarily enhances learning, and suggest that the benefits of multimodal instruction may depend on the balance between representational enrichment and cognitive load. More generally, they highlight important constraints on the implementation of multimodal learnings in educational practices, particularly in early literacy instruction.</p>