Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 12 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

*From a prompt to a fine-tuned, aligned, monitored assistant: every technique for adapting an open-weights LLM to your domain, on a single GPU, with reproducible numbers and the honesty to show when a technique does not win.*

![From a base model (Qwen3-4B) through the adaptation spectrum (prompting, RAG, LoRA/QLoRA, full SFT, distillation, DPO) to a focused domain assistant that is cheaper, private, and monitored, with every step running on a single GPU.](assets/book-hero.png)
![The whole-book arc as one diagram: a base model (Qwen3-4B) moves left to right through the adaptation spectrum with increasing depth (few-shot/RAG in chapter 4, LoRA/QLoRA in chapter 5, full SFT in chapter 6, distillation in chapter 7, DPO alignment in chapter 8) into a focused domain assistant that is cheaper, private, and monitored. The spectrum sits on a data foundation (chapter 3: quality, attribution, splits) and under an operations layer (chapter 9: monitor, drift, rollback, retrain), and every step runs on a single GPU with reproducible, measured results.](assets/book-hero.png)

Welcome to the code repository for **LLM Customization and Fine-Tuning: Adaptation, Distillation, and Alignment** (Manning Publications).

Expand Down Expand Up @@ -39,7 +39,7 @@ You need working Python and comfort at the command line; the book teaches the LL

- Choose the right adaptation technique (prompting/RAG vs LoRA/QLoRA vs full SFT vs distillation vs DPO) with an explicit cost, latency, privacy, and ROI framework.
- Fine-tune an LLM end to end with LoRA and QLoRA: data prep, training, evaluation, and inference.
- Build a training-data pipeline: curate and quality-gate real data, generate synthetic data with a frontier teacher, and track lineage and versioning.
- Build a training-data pipeline: curate and quality-gate real data, generate teacher-model training data (a local teacher in the book for reproducibility, a frontier API teacher in production), and track lineage and versioning.
- Distill a smaller, cheaper student from a stronger teacher, and align a model with DPO, with a safety-regression check at each step.
- Operate models in production: a model and data registry, drift detection with canary prompts, rollback procedures, a red-team safety monitor, and an outcome-based retraining cadence.

Expand All @@ -48,7 +48,7 @@ You need working Python and comfort at the command line; the book teaches the LL
- **The vendor-neutral, do-it-yourself counterpart to managed fine-tuning.** As the major vendors turn enterprise fine-tuning into a managed service, this book teaches the underlying methods and the operations layer those services abstract away, so you can run them on your own models and your own hardware and weigh build versus buy with real numbers.
- **The full technique spectrum in one running example.** A single fictitious enterprise (Contoso) and its IT help desk is threaded from prompting all the way through alignment and operations, so you see the same problem solved at every rung, with an explicit cost, latency, privacy, and ROI framework for choosing between the techniques.
- **Runnable and reproducible on a single GPU.** Every chapter runs on one GPU (LoRA and QLoRA on a modest consumer card; full fine-tuning and DPO on a single 24 GB card) and reproduces the book's published numbers within run-to-run variance. Validated on NVIDIA, AMD, and Apple Silicon.
- **Honest engineering.** The book shows when a technique does *not* win (for example, where DPO matches SFT on objective accuracy), anchors claims to real cost economics, and runs a safety-regression check at each step.
- **Honest engineering, in the open.** The book shows when a technique does *not* win (for example, where DPO matches SFT on objective accuracy), anchors claims to real cost economics, and runs a safety-regression check at each step. The code, the trained models, and the training and evaluation logs are all public, including the experiments that did not work, so you can verify every result and learn as much from what failed as from what worked.
- **The operational layer most LLM books skip.** Drift detection, versioning, rollback, and continuous safety monitoring, the part that decides whether a fine-tuned model survives past launch.

It uses a small open-weights model (Qwen3-4B) so every result reproduces on modest hardware, but the techniques themselves (LoRA, QLoRA, SFT, distillation, DPO) are model-size-agnostic and apply unchanged to larger frontier models, where scaling up is a matter of more memory and compute, not different methods.
Expand Down Expand Up @@ -100,15 +100,15 @@ The builder needs `beautifulsoup4` and `datasets` (both in the base install). Pe

| Chapter and topic | What you build |
|---|---|
| **[Chapter 1: Why Model Adaptation?](code/chapter01/README.md)** | A reproducibility script for the §1.6 sidebar. Runs the same prompt through base Qwen3-4B, the Chapter 5 LoRA adapter, and the Chapter 6 SFT model side by side; degrades gracefully if the later-chapter artifacts are not yet built. |
| **[Chapter 2: How Do I Do Model Adaptation?](code/chapter02/README.md)** | A five-step LoRA fine-tuning quickstart on Qwen3-4B-Instruct-2507 using a 40-example slice of the IT-support dataset (TRL's `SFTTrainer` plus PEFT): dataset prep, LoRA training, generation, and adapter save. Runs in under 10 minutes on a 12 GB GPU, and on Apple Silicon via MPS. |
| **[Chapter 3: What Data Do I Need?](code/chapter03/README.md)** | Data-quality experiment that trains the same model on four versions of Financial PhraseBank and compares results on a held-out test set; a six-step synthetic data generation pipeline (load → prompt → generate → quality-gate → distribution-check → mix-and-save) using a frontier teacher; and a standalone `DatasetManifest` module for content hashing, lineage tracking, and retention scheduling. |
| **[Chapter 4: In-Context Learning, Few-Shot, and RAG](code/chapter04/README.md)** | Few-shot ticket classifier, prompt validator with run-to-run variability measurement, minimal RAG pipeline (50 lines), and a Precision@k / Recall@k / Hit@1 retrieval evaluator. CPU-friendly; GPU optional. |
| **[Chapter 5: Parameter-Efficient Fine-Tuning (LoRA and QLoRA)](code/chapter05/README.md)** | LoRA and QLoRA adapters trained on the IT-support dataset (real Stack Exchange IT Q&A plus a Dolly slice) with Qwen3-4B-Instruct-2507, evaluated against the base model with per-category Token-F1 and a safety regression suite. |
| **[Chapter 6: Supervised Fine-Tuning (SFT)](code/chapter06/README.md)** | A full-parameter SFT of Qwen3-4B-Instruct-2507 on the IT-support dataset, with overfit monitoring, three-way base-vs-LoRA-vs-SFT comparison, behavioral tests, and a separate safety regression suite. |
| **[Chapter 7: Knowledge Distillation](code/chapter07/README.md)** | Black-box distillation from the chapter 6 SFT teacher into a chapter 5-style LoRA student, with quality filtering, three-way base-vs-teacher-vs-student evaluation, safety robustness check, and an optional OpenRouter-backed SFT-vs-frontier-API comparison. |
| **[Chapter 8: DPO and Advanced Alignment](code/chapter08/README.md)** | Preference-optimisation of the chapter 6 SFT model using TRL's `DPOTrainer`; three-way base-vs-SFT-vs-DPO comparison; safety regression after DPO. |
| **[Chapter 9: Managing Model Evolution, Drift, and Versioning](code/chapter09/README.md)** | A JSON-backed model registry, a TF-IDF drift detector, a simulated rollback workflow, a canary-prompt monitor, and a red-team safety monitor with per-category alerting. |
| **[Chapter 1: Why model adaptation?](code/chapter01/README.md)** | A reproducibility script for the §1.6 sidebar. Runs the same prompt through base Qwen3-4B, the Chapter 5 LoRA adapter, and the Chapter 6 SFT model side by side; degrades gracefully if the later-chapter artifacts are not yet built. |
| **[Chapter 2: How to do model adaptation](code/chapter02/README.md)** | A five-step LoRA fine-tuning quickstart on Qwen3-4B-Instruct-2507 using a 40-example slice of the IT-support dataset (TRL's `SFTTrainer` plus PEFT): dataset prep, LoRA training, generation, and adapter save. Runs in under 10 minutes on a 12 GB GPU, and on Apple Silicon via MPS. |
| **[Chapter 3: What data do I need for model adaptation?](code/chapter03/README.md)** | Data-quality experiment that trains the same model on four versions of Financial PhraseBank and compares results on a held-out test set; a six-step synthetic data generation pipeline (load → prompt → generate → quality-gate → distribution-check → mix-and-save) using a frontier teacher; and a standalone `DatasetManifest` module for content hashing, lineage tracking, and retention scheduling. |
| **[Chapter 4: In-context learning, few-shot, and RAG](code/chapter04/README.md)** | Few-shot ticket classifier, prompt validator with run-to-run variability measurement, minimal RAG pipeline (50 lines), and a Precision@k / Recall@k / Hit@1 retrieval evaluator. CPU-friendly; GPU optional. |
| **[Chapter 5: Parameter-efficient fine-tuning: LoRA and QLoRA](code/chapter05/README.md)** | LoRA and QLoRA adapters trained on the IT-support dataset (real Stack Exchange IT Q&A plus a Dolly slice) with Qwen3-4B-Instruct-2507, evaluated against the base model with per-category Token-F1 and a safety regression suite. |
| **[Chapter 6: Supervised fine-tuning: maximum expressiveness](code/chapter06/README.md)** | A full-parameter SFT of Qwen3-4B-Instruct-2507 on the IT-support dataset, with overfit monitoring, three-way base-vs-LoRA-vs-SFT comparison, behavioral tests, and a separate safety regression suite. |
| **[Chapter 7: Knowledge distillation: capturing frontier model intelligence](code/chapter07/README.md)** | Black-box distillation from the chapter 6 SFT teacher into a chapter 5-style LoRA student, with quality filtering, three-way base-vs-teacher-vs-student evaluation, safety robustness check, and an optional OpenRouter-backed SFT-vs-frontier-API comparison. |
| **[Chapter 8: Preference optimization: teaching your model to judge](code/chapter08/README.md)** | Preference-optimisation of the chapter 6 SFT model using TRL's `DPOTrainer`; three-way base-vs-SFT-vs-DPO comparison; safety regression after DPO. |
| **[Chapter 9: Managing model evolution, drift, and versioning](code/chapter09/README.md)** | A JSON-backed model registry, a TF-IDF drift detector, a simulated rollback workflow, a canary-prompt monitor, and a red-team safety monitor with per-category alerting. |

**Start here:**
1. [code/README.md](code/README.md): set up your Python environment and install the package.
Expand Down
Binary file modified assets/book-hero.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading