Skip to content

v0.11.0 — Instruction fine-tuning (mini chat)

Choose a tag to compare

@cattolatte cattolatte released this 07 Jul 01:18
ff377de

Turns a pretrained base decoder into a small instruction-following chat model, from scratch: a shared chat template, supervised fine-tuning with response-only loss masking (the -100 convention, so the existing trainer is unchanged), a bundled instruction set, a fine-tune script, and zenith chat --instruct (stops at EOS).

Honest model card: it nails trained instructions and close paraphrases and stops cleanly, but memorises more than it generalises at ~10M params / 84 examples — the value is a correct, legible SFT pipeline. Includes phase-11 design doc + ADR-0011.