Transformer-Based Multimodal Image Fusion in Healthcare: A Comprehensive Survey of Architectures, Fusion Strategies, and Clinical Applications

Abstract

Transformer architectures have changed how multimodal medical image fusion works. Convolutional neural network (CNN)-based methods are constrained to local receptive fields; transformers use self-attention and cross-modal interactions to capture dependencies across the full image, which matters when aligning magnetic resonance imaging (MRI), computed tomography (CT), positron emission tomography (PET), and ultrasound—modalities that differ substantially in resolution, contrast, and the anatomical features they emphasize. This survey organizes transformer-based fusion models for healthcare along four axes:modality, pairing, fusion level, and attention and architecture design. Four representative models—MACTFusion, ECFusion, MATR, and DFENet—are examined in detail, with attention to how each handles anatomical preservation,semantic reasoning, and clinical interpretability. Fusion strategies fall into pixel-level, feature-level, segmentation, and neurodegeneration staging, and the survey covers the evaluation metrics and benchmark datasets used to assess performance across these tasks. A number of problems have not been resolved. Data scarcity and modality heterogeneity continue to limit generalization; interpretability remains shallow in most deployed architectures; and clinical environments impose constraints—compute budgets, latency requirements, and regulatory oversight—that most published models are not designed for. Work on lightweight vision transformers, language-guided prompting, 3D and temporal extensions, and federated learning addresses some of these gaps, though none yet offers a comprehensive solution. The survey aims to give researchers and clinicians a structured entry point into this area.

Received: 20 November 2025 | Revised: 13 July 2026 | Accepted: 28 August 2026

Conflicts of Interest

The authors declare that they have no conflicts of interest in this work.

Data Availability Statement

Data sharing is not applicable to this article, as no new data were created or analyzed in this study. The benchmark datasets discussed in this survey (BRATS, ADNI, CHAOS, OASIS, ISLES, TCIA, IXI, and EyePACS) are third-party resources, described in their original publications and listed in Section 5.1.1 and Table 8. Most are publicly accessible; a few, including ADNI and TCIA, require researchers to register with the data provider and sign a data use agreement before access is granted. Access to all datasets is subject to the terms set by their respective providers; readers should consult the original sources cited here for current access procedures.

Author Contribution Statement

Nalini S Jagtap: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization. Dilip Kumar Jang Bahadur Saini: Conceptualization, Methodology, Software, Validation, Formal analysis, Resources, Writing – review & editing, Supervision. Prasun Chakrabarti: Conceptualization, Methodology, Formal analysis, Writing –review & editing, Supervision.