PANDIA: Personalized neuro-symbolic multimodal fusion for interpretable neonatal pain assessment

Abstract

Effective pain assessment in infants aged 0–3 months is a critical challenge in neonatal intensive care units (NICUs) and family medicine clinics, where self-reporting is impossible and current observational tools remain subjective and inconsistent. This paper presents PANDIA (Personalized Adaptive Neuro-symbolic Data-fusion for Infant Assessment), a novel multimodal AI system that combines hierarchical representation learning, graph-based inter-modal reasoning, meta-learning personalization, and symbolic concept-bottleneck explanations for robust infant pain assessment. Unlike transformer-centric approaches, PANDIA employs lightweight CNN/TCN backbones with a graph neural network for inter-modal fusion, achieving clinical interpretability through explicit concept bottlenecks and symbolic reasoning. Our federated learning framework enables privacy-preserving multi-site collaboration while meta-learning adaptation provides personalized assessment with minimal per-infant data. Evaluated on 2,847 infants across four datasets, PANDIA achieves 87.3% accuracy with 92.1% clinician acceptance rate for explanations, achieving a 12.4% accuracy improvement over the best baseline, consistent across all four datasets and an independent out-of-distribution test set, while maintaining fewer than 30M parameters for edge deployment. The proposed system offers a structured and interpretable step toward deploying explainable AI in early-life pain management, with potential to improve care quality and support medical decision-making. Key limitations include the retrospective validation design, dataset heterogeneity across collection sites, and the need for prospective clinical trials before deployment in live clinical settings. All code, trained models, preprocessing pipelines, and supplementary materials are fully publicly available without restriction at:. The NICU-MM dataset is available upon request subject to an ethical data use agreement; the access procedure is detailed in. Author summary Assessing pain in newborns is challenging because they cannot communicate their discomfort. Current methods rely on observing behaviors such as crying or facial expressions, but these observations can be inconsistent and subjective, leading to undertreated or overtreated pain with long-term developmental consequences. PANDIA is an AI-powered tool designed to help doctors and nurses accurately and continuously assess pain in newborns aged 0–3 months using video, audio, and physiological signals from standard NICU equipment. Unlike prior approaches that rely on large, computationally heavy models, PANDIA uses lightweight neural network components combined with symbolic reasoning to provide clear, clinically grounded explanations that healthcare providers can understand and trust. The system adapts to each infant’s unique pain expression patterns with as few as five labeled examples, and protects patient privacy through a federated learning framework that trains across multiple hospital sites without sharing raw data. Evaluated across 2,847 infants from four datasets spanning four continents, PANDIA achieves 87.3% accuracy with a 92.1% clinician acceptance rate, outperforming current methods while remaining suitable for deployment on hospital-edge devices. We hope this work contributes to better pain management and improved outcomes for vulnerable neonatal populations worldwide. summary