Incorporating brain-inspired mechanisms for multimodal learning in artificial intelligence

Source: PubMed Central Open Access, NCBI / U.S. National Library of Medicine

Multimodal learning enhances the perceptual ability of intelligent systems by integrating information across sensory modalities. However, most artificial intelligence approaches still rely on static fusion schemes and do not account for the dynamic nature of multisensory integration observed in the brain. In biological systems, the principle of inverse effectiveness states that weaker unimodal cues contribute more strongly to multisensory integration, whereas strong cues reduce the relative benefit of fusion. Motivated by this mechanism, we investigate the relationship between multimodal outputs and modality-specific information and introduce an inverse effectiveness–driven multimodal fusion (IEMF) strategy. Integrating IEMF into neural networks yields more adaptive fusion behavior, improved accuracy, and substantial gains in computational efficiency. We validate the generality of IEMF across audiovisual perception, vision-language understanding, and trimodal sentiment analysis, demonstrating consistent improvements over state-of-the-art baselines in classification, continual learning, and question answering. The approach also transfers across architectures, including both artificial neural networks and spiking neural networks. Biologically inspired inverse effectiveness enables more adaptive and efficient multimodal learning across diverse sensory tasks. teaser