mlx distillation explained.stub

---

id: mlx-distillation-explained name: MLX Distillation Explained repo_path: /sessions/charming-sleepy-ritchie/repos/mlx-distillation-explained remote_url: https://github.com/sw30labs/mlx-distillation-explained.git primary_language: Python framework: MLX + LangGraph ingested: 2026-04-11 last_commit_date: 2026-04-11

MLX Distillation Explained

Purpose

Educational PoC demonstrating model distillation from Claude Sonnet (teacher) to Llama 3.1 8B (student) via LoRA fine-tuning on Apple Silicon. Full pipeline with verification gates between steps.

Architecture

LangGraph-orchestrated 5-step pipeline (run_pipeline.py) with verification gates:

1. Extract Teacher Examples: Query Claude Sonnet for high-quality examples
2. Eval Baseline: Evaluate student (Llama 3.1 8B) performance before distillation
3. LoRA Fine-Tune: Apply LoRA adapter on Apple Silicon
4. Eval Adapted: Evaluate student performance post-fine-tuning
5. Benchmark Report: Generate comparison report

Each step has individual scripts for standalone execution.

Key Technologies

• Teacher Model: Claude Sonnet via Anthropic API
• Student Model: Llama 3.1 8B via mlx-lm
• Fine-Tuning: LoRA adapters via MLX
• Orchestration: LangGraph StateGraph
• Verification: Gates between steps

Dependencies

mlx, mlx-lm, mlx-vlm, anthropic, python-dotenv, pyyaml, langgraph, langchain-core, openpyxl

Tags

python, mlx, apple-silicon, local-inference, lora, distillation, langgraph

Cross-references

• qwenbench-mlx — benchmarking MLX models