Gemini_Generated_Image_e5v597e5v597e5v5

From Drones to Decisions: LLM-Based Crop Health Insights from RGB and Multispectral Data

by Precision Agriculture

From Drones to Decisions: LLM-Based Crop Health Insights from RGB and Multispectral Data

This project addresses one of the most pressing challenges in Pakistani agriculture: the lack of timely, automated systems for monitoring crop health and growth. While drones equipped with RGB and multispectral cameras are increasingly available, most existing tools stop at producing raw imagery or vegetation indices, leaving farmers without actionable insights. This project bridges that gap by creating an AI-powered pipeline that ingests UAV imagery (either RGB, multispectral, or both) and transforms it into farmer-friendly recommendations. By training separate models for each data type and then introducing an attention-based fusion mechanism, the system flexibly adapts to different sensor setups while ensuring accurate phenology detection and yield forecasting. Instead of overwhelming farmers with technical details, the outputs are passed through a language model that translates them into clear information.

High-resolution RGB data is processed with CNNs to extract texture and spatial features, while multispectral imagery is converted into vegetation indices such as NDVI, NDRE, and GNDVI, then fed into structured classifiers. These complementary representations are integrated using a cross-attention fusion model, which learns to prioritize the most informative signals for each phenological stage. Predictions are augmented by metadata, enabling the generation of temporal growth maps and yield estimates across different fields. To make these technical outputs truly usable, a lightweight large language model (LLM) formats them into natural language reports. For example, instead of saying “NDVI decreased by 15%”, the system might explain, “The crop canopy is thinning earlier than expected, suggesting possible water stress”.

The impact of this project extends beyond proof-of-concept research. Farmers and agronomists gain an accessible web interface where they can upload drone imagery, receive instant model predictions, and view interpretive reports tailored to their crops. By enabling early detection of stress, accurate stage tracking, and reliable yield forecasting, the system helps reduce input waste, optimize irrigation and fertilization schedules, and ultimately increase productivity. For Pakistan, where agriculture contributes nearly 20% of GDP but still faces massive inefficiencies due to low adoption of digital tools, this project demonstrates how modern AI can be adapted to local constraints and translated into practical, farmer-readable outputs.

Faculty

Students

  • Saleha Zainab Fatima
  • Munazza Raees Butt

Publications

  • Fatima, S.Z., Raees, M., Athar, U., Asif, M.D.A., Zafar, Z., Khurshid, H., Berns, K. and Fraz, M.M., 2025, November. Semi-Supervised Contrastive Representation Learning for Sunflower Phenology Estimation. In 2025 20th International Conference on Emerging Technologies (ICET) (pp. 1-7). IEEE. https://doi.org/10.1109/ICET66147.2025.11321433
Gemini_Generated_Image_b0z1nvb0z1nvb0z1

HunarmandAI: Culturally-Aware Intelligent Tutoring for Low-Literacy Urdu Speakers

by Intelligent Systems

HunarmandAI: Culturally-Aware Intelligent Tutoring for Low-Literacy Urdu Speakers

HunarmandAI is an intelligent tutoring system designed specifically for Pakistani auto electricians who often struggle with limited access to training resources due to low literacy levels and language barriers. While most digital platforms and AI tools are dominated by English or highly formal Urdu, the everyday language spoken in workshops is far more informal, localized, and mixed with English technical terms. HunarmandAI bridges this gap by introducing a bilingual Urdu-English small language model (SLM) that can understand and respond in a way that feels natural to electricians. By embedding specialized automotive electrical knowledge directly into the model, the project creates a culturally and linguistically accessible tutor that works entirely offline, ensuring usability even in areas with little or no internet connectivity.

This project pushes the boundaries of what small language models can achieve by focusing on reasoning, accessibility, and edge deployment. Unlike general-purpose chatbots, HunarmandAI is fine-tuned with instruction-following datasets, step-by-step reasoning examples, and domain-specific content derived from real-world resources used by technicians. Through methods like instruction tuning, chain-of-thought optimization, and reinforcement learning with human feedback, the model is trained not only to provide correct answers but also to explain its reasoning in clear, easy-to-follow steps. This makes the tutor much more than just a Q&A tool. It becomes a practical guide that teaches problem-solving, repair techniques, and diagnostic skills in a way that mirrors how electricians actually work.

HunarmandAI has a strong social and economic impact as well. It empowers auto electricians from lower and lower-middle-class backgrounds to learn independently, improve their craft, and enhance their earning potential without relying on costly training centers or constant internet access. At the same time, it contributes to AI research by demonstrating how small language models can be adapted for reasoning-rich, domain-specific tasks on low-power devices. By releasing the model, dataset, and mobile app openly, the project also lays a foundation for future culturally-aware AI tutors in other vocational fields, advancing digital inclusion and supporting sustainable skill development in underserved communities.

Faculty

Students

  • Muhammad Nabeel
  • Hafiza Adeela Arif
  • Abubakar Khan
Gemini_Generated_Image_l9fsgbl9fsgbl9fs

MedLingo: Medical LLM with Mixture of Experts

by Healthcare

MedLingo: Medical LLM with Mixture of Experts

MedLingo is a clinical decision-support system designed to bring specialization and precision into medical AI. Unlike general-purpose language models that try to cover everything but often lack depth in critical areas, MedLingo is built around a Mixture-of-Experts framework. Instead of relying on a single, monolithic model, it coordinates multiple smaller, domain-specific language models—each trained in a specialty such as ophthalmology, neurology, or oncology. This modular setup allows MedLingo to act more like a team of medical experts: when a clinician presents a case, the system routes the input to the most relevant specialists, who then collaborate and reconcile their findings. By integrating multimodal data (clinical text, lab results, medical imaging), the platform delivers context-aware analysis that mirrors the way real doctors synthesize evidence.

This project combines natural language understanding with medical image encoding and structured data processing, ensuring no critical detail is left out of the diagnostic process. The system is evaluated on a wide range of benchmarks, from MIMIC-III and PubMedQA for clinical text to CheXpert and ISIC for imaging, ensuring that its performance is robust across diverse data types. Its architecture is carefully designed for efficiency: the Mixture-of-Experts controller dynamically selects the most relevant models for each query, reducing unnecessary computation and maintaining low latency. This makes MedLingo not only accurate but also scalable, allowing new expert models to be added over time without overhauling the system.

The impact of MedLingo extends beyond technical novelty. For healthcare professionals, it offers faster, more reliable diagnostic support that could reduce errors, accelerate clinical decision-making, and improve patient outcomes. For researchers, it provides a modular blueprint for how AI can be structured in medicine, demonstrating how collaboration between expert models can enhance reasoning in ways a single system cannot. And for the broader community, the project contributes open-access code, datasets, and methodologies, fostering transparency and future innovation in medical AI. By making clinical decision-support systems more specialized, adaptable, and multimodal, MedLingo represents a step toward AI that works not as a black box, but as an intelligent and trusted partner in healthcare.

Faculty

Students

  • Muhammad Samiullah
  • Sarmad Sultan
  • Ahmad Jan
Gemini_Generated_Image_w8kd0gw8kd0gw8kd

LiDAR-Based Plant Phenotyping for Precision Agriculture

by Precision Agriculture

LiDAR-Based Plant Phenotyping for Precision Agriculture

This project tackles one of the biggest bottlenecks in modern crop science: how to measure plant traits quickly, accurately, and at scale. Traditional phenotyping is still largely manual. Researchers take rulers and calipers into fields, a process that is slow, destructive, and vulnerable to human error. This project proposes a smarter alternative by harnessing LiDAR sensors to capture rich 3D representations of plants and applying deep learning models to directly interpret that structural data. Instead of reducing plants to simplistic measurements like height or canopy width, the system learns to estimate key biological traits such as above-ground biomass, canopy volume, and other indicators of crop performance. By developing a full software pipeline and integrating both open datasets and custom LiDAR scans, this project aims to create a scalable, non-destructive tool that makes high-throughput phenotyping practical for breeders and agronomists.

The technical backbone of the project is a novel deep learning model, nicknamed AgriFormer, designed to predict above-ground biomass directly from 3D point cloud data. The architecture processes raw LiDAR scans through a multiscale encoder-decoder and then reconstructs dense plant structures to deliver highly accurate predictions. Crucially, this avoids the need for hand-crafted features or voxelization, which limit flexibility and accuracy. Early experiments using open-source datasets show promising results with lower error rates compared to conventional approaches. Alongside this, we build an in-house dataset using custom 2D LiDAR hardware mounted on a UAV platform, generating full 3D reconstructions of target crops.

The significance of this project lies in its potential to reshape how agricultural research is conducted. By lowering the cost and effort of high-fidelity phenotyping, it enables crop scientists to study larger populations with greater accuracy, accelerating breeding programs aimed at higher yields and climate resilience. The methodology also lowers the barrier to entry for labs with limited resources, since the conversion of 2D LiDAR scans into 3D point clouds offers a cost-effective alternative to expensive commercial systems. The pipeline and datasets produced in this work can power a new wave of data-driven agriculture, helping farmers and researchers make smarter decisions about crop management and varietal selection.

Faculty

Students

  • Abdul Wahab
  • Faareh Ahmed
  • Malik Shahzaib Khan

Publications

  • Khan, M.S., Ahmed, F., Wahab, A., Zafar, Z., Berns, K. and Fraz, M.M., 2025, October. AgriFormer: Advancing 3D LiDAR-based Biomass Prediction through Hierarchical Feature Learning. In 2025 IEEE/ACS 22nd International Conference on Computer Systems and Applications (AICCSA) (pp. 1-6). IEEE. https://doi.org/10.1109/AICCSA66935.2025.11315189
Gemini_Generated_Image_j0k42aj0k42aj0k4

GlacioVision: Modeling Glacier Retreat and Water Security in Pakistan Through Remote Sensing Technologies

by Environment & Sustainability

GlacioVision: Modeling Glacier Retreat and Water Security in Pakistan Through Remote Sensing Technologies

GlacioVision is an intelligent framework to forecast glacier retreat and evaluate its impact on water resources. Pakistan’s glaciers, particularly in the Karakoram and Hindu Kush, serve as vast natural reservoirs that feed the Indus River system and sustain agriculture, hydropower, and drinking water supplies for millions. But rising temperatures and erratic weather patterns are rapidly destabilizing these fragile ice bodies, leading to increased flood risks, seasonal water variability, and long-term freshwater shortages. GlacioVision tackles this challenge by combining multi-year satellite imagery with climate and meteorological data to model how glaciers evolve over time, offering both scientific insights and practical tools for water and disaster management.

At the heart of the project is a deep learning model designed to jointly predict glacier extent and surface elevation. Unlike traditional methods that treat these aspects separately, GlacioVision integrates Sentinel-1 radar imagery, Sentinel-2 optical data, ICESat-2 elevation points, and reanalysis climate variables into a single predictive system. The model leverages convolutional neural networks to extract spatial patterns, long short-term memory modules to capture temporal dynamics, and multilayer perceptrons to embed atmospheric influences. This multimodal, temporally-aware design allows the framework to generate high-resolution predictions of glacier masks and digital elevation models (DEMs), producing a comprehensive view of both surface retreat and volumetric changes. By learning from historical sequences of glacier behavior, the system can forecast near-future changes, providing a critical early warning mechanism for water security planning and climate adaptation.

GlacioVision’s predictive outputs can support government agencies in designing resilient water infrastructure, help disaster managers anticipate glacial lake outburst floods in vulnerable valleys, and guide farmers and hydropower operators in adapting to shifting river flows. The framework’s reliance on open satellite and climate datasets makes it scalable and replicable in other glacier-dependent regions of the world. In Pakistan, however, the project holds especially urgent significance because it provides an accessible way to understand glacier decline.

Faculty

Students

  • Muhammad Sarmad Saleem
  • Abubakar Imran
  • Ali Haider
Gemini_Generated_Image_kixmgmkixmgmkixm

Transforming Clinical Decision-Making: Predicting Antimicrobial Resistance from MALDI-TOF Data

by Healthcare

Transforming Clinical Decision-Making: Predicting Antimicrobial Resistance from MALDI-TOF Data

Antimicrobial resistance (AMR) has become one of the biggest threats to modern medicine, with delays in diagnosis directly costing lives. Conventional Antimicrobial Susceptibility Tests (ASTs) can take up to three days and require highly trained microbiologists, which makes them inaccessible in many hospitals, especially in low-resource regions like Pakistan. To overcome this barrier, our project harnesses the power of deep learning to predict resistance profiles directly from MALDI-TOF mass spectrometry data within minutes. Instead of waiting for cultures to grow, doctors can obtain resistance predictions almost instantly, enabling faster clinical decision-making and improving patient outcomes.

Our approach combines species-specific multi-label convolutional neural networks (CNNs), transfer learning, and continual learning strategies to create a scalable and adaptable pipeline. Starting with over 300,000 spectra from Swiss and German hospitals, the models are trained to recognize resistance signatures for critical pathogens such as Escherichia coli, Klebsiella pneumoniae, Staphylococcus aureus, and Pseudomonas aeruginosa. Through transfer learning, the models generalize across different hospital datasets, while continual learning mechanisms allow them to adapt to new bacterial strains and evolving antibiotic resistance patterns without catastrophic forgetting. This adaptability ensures that the system remains clinically relevant even as microbial landscapes shift over time.

This project not only delivers high-performing models benchmarked against existing state-of-the-art systems but also lays the foundation for real-world deployment. By integrating the continual learning framework into a lightweight web API, the system can provide resistance predictions in real time, making it practical for use in hospitals worldwide. Beyond its direct clinical utility, the project demonstrates how AI can bridge the gap between cutting-edge microbiological research and frontline healthcare needs. It introduces a reproducible, generalizable, and future-proof pipeline that could transform how infections are diagnosed, ultimately reducing diagnostic delays, improving antibiotic stewardship, and strengthening global efforts to combat AMR.

Faculty

Students

  • Hasaan Hamid
  • Suman Kumari
  • Afra
Gemini_Generated_Image_epeagdepeagdepea

XMedFusion: Agentic and Expert-Driven AI for Cross-Modal Medical Report Generation

by Healthcare

XMedFusion: Agentic and Expert-Driven AI for Cross-Modal Medical Report Generation

XMedFusion is an advanced AI system that reimagines how medical reports are created from imaging data. Unlike conventional models that struggle to adapt across modalities, XMedFusion employs a Mixture of Experts (MoE) architecture where specialized branches are trained for X-rays, CTs, and MRIs. A lightweight gating mechanism intelligently routes each case to the right experts, ensuring that the resulting reports are accurate, modality-aware, and clinically coherent. The system goes beyond automation by grounding its outputs in trusted biomedical literature such as PubMed, creating reports that are not only precise but also transparent in their reasoning.

At the core of XMedFusion is an agentic pipeline that treats report generation as a structured process rather than a black-box prediction task. Each expert agent contributes targeted insights, which are then synthesized into a comprehensive medical narrative enriched with explainability features like Grad-CAM overlays and attention maps. Radiologists remain integral to the loop through a human-in-the-loop (HIL) dashboard, where they can review, edit, and refine the AI-generated drafts. These corrections feed back into the system for continuous learning, enabling XMedFusion to evolve in alignment with clinical standards and real-world requirements.

The project’s broader vision is to ease the workload of radiologists while ensuring faster and safer diagnostic outcomes. By integrating explainable AI, external knowledge grounding, and active expert feedback, XMedFusion provides a scalable solution that can adapt to new modalities, diverse datasets, and varying clinical contexts. Its impact is twofold: scientifically, it demonstrates how domain knowledge and multi-expert coordination can be embedded into deep learning pipelines; practically, it offers healthcare providers a trustworthy tool that accelerates diagnosis, reduces errors, and makes high-quality medical insights more accessible, especially in resource-limited settings.

Faculty

Students

  • Arham Haroon
  • Hamza Riaz
  • Maha Baig
hec award

HEC Honors Prof Dr Muhammad Moazam Fraz for Global Research Influence

by News
HEC Recognition

HEC Honors Prof. Dr. Moazam Fraz for Global Research Influence

Prof. Dr. Muhammad Moazam Fraz has been honored by the Higher Education Commission (HEC) of Pakistan for his exceptional global research influence and valuable contribution to the advancement of knowledge in his field.

tamgha

Prof Dr Moazam Fraz Honored with Prestigious Tamgha-i-Imtiaz

by News
Pakistan Civil Award

Prof Dr Moazam Fraz Honored with Prestigious Tamgha-i-Imtiaz

Prof Dr Muhammad Moazam Fraz, Associate Dean (Computing) SEECS, was awarded the highly prestigious Tamgha-i-Imtiaz by the President of the Islamic Republic of Pakistan on May 13, 2026.