XMedFusion: Agentic and Expert-Driven AI for Cross-Modal Medical Report Generation

XMedFusion is an advanced AI system that reimagines how medical reports are created from imaging data. Unlike conventional models that struggle to adapt across modalities, XMedFusion employs a Mixture of Experts (MoE) architecture where specialized branches are trained for X-rays, CTs, and MRIs. A lightweight gating mechanism intelligently routes each case to the right experts, ensuring that the resulting reports are accurate, modality-aware, and clinically coherent. The system goes beyond automation by grounding its outputs in trusted biomedical literature such as PubMed, creating reports that are not only precise but also transparent in their reasoning.

At the core of XMedFusion is an agentic pipeline that treats report generation as a structured process rather than a black-box prediction task. Each expert agent contributes targeted insights, which are then synthesized into a comprehensive medical narrative enriched with explainability features like Grad-CAM overlays and attention maps. Radiologists remain integral to the loop through a human-in-the-loop (HIL) dashboard, where they can review, edit, and refine the AI-generated drafts. These corrections feed back into the system for continuous learning, enabling XMedFusion to evolve in alignment with clinical standards and real-world requirements.

The project’s broader vision is to ease the workload of radiologists while ensuring faster and safer diagnostic outcomes. By integrating explainable AI, external knowledge grounding, and active expert feedback, XMedFusion provides a scalable solution that can adapt to new modalities, diverse datasets, and varying clinical contexts. Its impact is twofold: scientifically, it demonstrates how domain knowledge and multi-expert coordination can be embedded into deep learning pipelines; practically, it offers healthcare providers a trustworthy tool that accelerates diagnosis, reduces errors, and makes high-quality medical insights more accessible, especially in resource-limited settings.

Faculty

Dr. Muhammad Moazam Fraz
Dr. Muhammad Moazam Fraz
Dr. Muhammad Naseer Bajwa
Dr. Muhammad Naseer Bajwa

Students

Arham Haroon
Hamza Riaz
Maha Baig