Gemini_Generated_Image_e5v597e5v597e5v5

From Drones to Decisions: LLM-Based Crop Health Insights from RGB and Multispectral Data

From Drones to Decisions: LLM-Based Crop Health Insights from RGB and Multispectral Data

This project addresses one of the most pressing challenges in Pakistani agriculture: the lack of timely, automated systems for monitoring crop health and growth. While drones equipped with RGB and multispectral cameras are increasingly available, most existing tools stop at producing raw imagery or vegetation indices, leaving farmers without actionable insights. This project bridges that gap by creating an AI-powered pipeline that ingests UAV imagery (either RGB, multispectral, or both) and transforms it into farmer-friendly recommendations. By training separate models for each data type and then introducing an attention-based fusion mechanism, the system flexibly adapts to different sensor setups while ensuring accurate phenology detection and yield forecasting. Instead of overwhelming farmers with technical details, the outputs are passed through a language model that translates them into clear information.

High-resolution RGB data is processed with CNNs to extract texture and spatial features, while multispectral imagery is converted into vegetation indices such as NDVI, NDRE, and GNDVI, then fed into structured classifiers. These complementary representations are integrated using a cross-attention fusion model, which learns to prioritize the most informative signals for each phenological stage. Predictions are augmented by metadata, enabling the generation of temporal growth maps and yield estimates across different fields. To make these technical outputs truly usable, a lightweight large language model (LLM) formats them into natural language reports. For example, instead of saying “NDVI decreased by 15%”, the system might explain, “The crop canopy is thinning earlier than expected, suggesting possible water stress”.

The impact of this project extends beyond proof-of-concept research. Farmers and agronomists gain an accessible web interface where they can upload drone imagery, receive instant model predictions, and view interpretive reports tailored to their crops. By enabling early detection of stress, accurate stage tracking, and reliable yield forecasting, the system helps reduce input waste, optimize irrigation and fertilization schedules, and ultimately increase productivity. For Pakistan, where agriculture contributes nearly 20% of GDP but still faces massive inefficiencies due to low adoption of digital tools, this project demonstrates how modern AI can be adapted to local constraints and translated into practical, farmer-readable outputs.

Faculty

Students

  • Saleha Zainab Fatima
  • Munazza Raees Butt

Comments are closed.