A Multi-Agent LLM-Orchestrated Platform for Automated Multi-Organ CT/MRI Segmentation with Natural Language Control

A next-generation medical imaging platform that combines AI-driven organ segmentation, LLM-based workflow orchestration, and natural language commands using voice or chat.

Built using Python + Gradio + Ollama (Llama3), this system supports both CT and MRI datasets and enables multi-organ 3D anatomical segmentation in a conversational way.
Ideal for radiology AI research, medical imaging workflows, visualization tools, and AI-powered clinical assistants.

---

🚀 Key Features

✅ Automated segmentation of major anatomical regions
✅ Supports Heart, Coronary Arteries, Abdomen Organs, Musculoskeletal (Thigh/Shoulder)
✅ Multi-Agent AI architecture for task routing and workflow automation
✅ Natural Language Control – run segmentation with chat or voice commands
✅ Local LLM Integration using Ollama + Llama3 8B
✅ GPU acceleration (CUDA by default, CPU fallback)
✅ Multi-label automatic merging for multiple structures
✅ 3D overlays + animated previews generated automatically
✅ Persistent case memory + session history
✅ Designed to extend easily (add liver, spine, pancreas, brain next)

---

🧱 System Architecture

flowchart TD User -->|Voice/Text| LLM_Agent LLM_Agent --> Orchestrator Orchestrator --> SegmentationAgent Orchestrator --> MergeAgent Orchestrator --> VisualizationAgent Orchestrator --> QCAgent SegmentationAgent -->|CT/MRI| MedImgProcessor MergeAgent --> UnifiedMasks VisualizationAgent --> OverlayImages QCAgent --> ReportSummary

🔧 Installation (Conda + GPU Recommended)

conda create -n medseg python=3.10 -y conda activate medseg

Core dependencies

pip install nibabel numpy pillow gradio sounddevice scipy python-dotenv

LLM

pip install ollama

Optional for GPU audio support

pip install torch torchaudio --index-url https://download.pytorch.org/whl/cu118

Install segmentation backend

pip install totalsegmentator==2.1.0 # Compatible engine (no CLI needed)

##▶️ Run Locally python app_totalseg_phase3.py

💬 Natural Language Commands

You can control segmentation using plain English instructions:

Command	Action
"Segment heart chambers"	Runs cardiac segmentation
"Extract coronary arteries"	Coronary segmentation
"Run abdomen organs"	Abdominal anatomy segmentation
"Do thigh muscles segmentation"	Muscle group segmentation
"Merge masks"	Combines multiple structure masks into one output
"Show results"	Displays overlays and segmentation previews
"Clean output and restart"	Resets system and clears outputs
"Save results"	Stores final segmentation and visual overlays

🔊 Voice Command Examples

You can also control the system hands-free using your microphone:

Voice Command Example	Action Triggered
"Start heart chamber segmentation"	Segments cardiac chambers
"Extract coronary arteries"	Runs coronary artery pipeline
"Process abdomen CT scan"	Segments liver, spleen, kidneys, etc.
"Segment thigh muscle region"	Runs musculoskeletal segmentation
"Show segmentation output"	Displays overlay results
"Save the results"	Saves masks + visual previews
"Restart and clear output"	Resets the session
"Help"	Displays help options