Unsloth integration: QLoRA + GRPO fine-tuning on AMD consumer GPUs

[kovtcharov]

Summary

Integrate Unsloth for efficient LoRA/QLoRA and GRPO fine-tuning on AMD consumer GPUs (Ryzen AI, Radeon). Enable users to fine-tune Qwen3-4B/8B models for specific agent tasks using minimal VRAM.

Why Unsloth

• 70% less VRAM — Qwen3-1.7B GRPO runs on just 5GB VRAM
• GRPO support — Primary RL algorithm, removes critic model entirely
• 500+ models — Including Qwen3 (GAIA's default model family)
• Unsloth Studio — No-code UI for non-ML-experts
• 100% offline — All training runs locally, no data leaves device
• ROCm support — Works on AMD GPUs

VRAM Requirements (practical)

Model	Training Type	VRAM Needed
Qwen3-1.7B	FP8 GRPO	~5 GB
Qwen3-4B	QLoRA GRPO	~4 GB
Qwen3-8B	QLoRA GRPO	~8 GB
Qwen3.5-14B	QLoRA SFT	~14 GB

Components

1. Unsloth backend — gaia finetune CLI command wrapping Unsloth
2. ROCm compatibility — Verify all training paths work on AMD GPUs
3. Pre-built training configs — Per-agent defaults (chat, code, email, etc.)
4. Model export — Save fine-tuned model as GGUF for Lemonade Server
5. Agent UI integration — "Improve this agent" button triggers fine-tuning

Dependencies

• Unsloth (pip install)
• ROCm / AMD GPU drivers
• Eval-to-training pipeline (for GRPO reward data)

Acceptance Criteria

• gaia finetune --agent chat --data eval_results.json works end-to-end
• QLoRA SFT training runs on AMD Radeon GPU
• GRPO training runs with Unsloth on AMD GPU
• Exported GGUF model loads in Lemonade Server
• Non-expert can trigger fine-tuning from Agent UI