v1.0.0 — First stable release

Zenith 1.0 — a from-scratch generative NLP library, now stable and on PyPI.

pip install "zenith-nlp[all]"

A complete, honest, from-scratch decoder-only LM stack:

• Modern architecture — Llama-style (RoPE, RMSNorm, SwiGLU) with an optional fused SDPA path; configurable GPT-2-style.
• Generation — greedy / temperature / top-k / nucleus / beam / streaming, plus greedy-exact speculative decoding.
• Training & scaling — causal-LM trainer, LoRA / AMP / DDP, a measured scaling study, and honest benchmarks (matches nanoGPT: 2.08 bits/char on tiny-shakespeare).
• Instruction tuning — response-only-masked SFT into a mini chat model (zenith chat --instruct), with a model card.
• int8 quantized inference, a vectorized from-scratch BPE, FastAPI serving + SSE, MLflow tracking, Hydra configs, and optional Polaris interop.

103 tests on Python 3.10–3.12, 11 ADRs, 11 design docs, benchmark figures. The public API follows semantic versioning from here.

Full history in CHANGELOG.md.

v0.13.1 — Demo GIF

A terminal-style demo GIF of the instruction-tuned model chatting, rendered from real greedy output with Pillow (no external recorders) and embedded in the README. Reproducible via scripts/render_demo_gif.py.

v0.13.0 — int8 quantized inference

zenith.quantize adds weight-only int8 quantization: QuantizedLinear + quantize_int8(model), ~4× smaller linear weights with greedy output unchanged on the 10.7M model. zenith generate --int8. Saves memory (no int8 matmul kernel, so not a CPU/MPS speedup) — honest scope.

v0.12.1 — Vectorized BPE training

BPETokenizer.train is now vectorized with numpy — ~8× faster than the naive Python loop (gap grows with corpus size), byte-level semantics and losslessness unchanged. numpy is imported lazily so encode/decode stay dependency-light.

v0.12.0 — Fused SDPA attention (opt-in)

Optional attention="sdpa" config uses PyTorch's fused scaled_dot_product_attention (RoPE + KV-cache + causal preserved), numerically equivalent to the eager path and ~1.28× faster on the 10.7M model. Default stays eager; old checkpoints load unchanged.

v0.11.0 — Instruction fine-tuning (mini chat)

Turns a pretrained base decoder into a small instruction-following chat model, from scratch: a shared chat template, supervised fine-tuning with response-only loss masking (the -100 convention, so the existing trainer is unchanged), a bundled instruction set, a fine-tune script, and zenith chat --instruct (stops at EOS).

Honest model card: it nails trained instructions and close paraphrases and stops cleanly, but memorises more than it generalises at ~10M params / 84 examples — the value is a correct, legible SFT pipeline. Includes phase-11 design doc + ADR-0011.

v0.10.0 — Speculative decoding

Greedy-exact speculative decoding, from scratch: a small draft model proposes tokens, the large target verifies them in one pass, and the output is byte-for-byte identical to greedy decoding on the target. SpeculativeStats reports acceptance rate and the target-forward speedup; KVCache.truncate rolls back rejected drafts.

Real benchmark (0.6M draft + 10.7M target, tiny-shakespeare): ~3× fewer target forward passes at identical output. Honest caveat in BENCHMARKS — forwards ≠ wall-clock at this tiny scale (~1.05–1.08× on MPS); the win grows with the target/draft size gap. Includes phase-10 design doc + ADR-0010.

v0.9.2 — Benchmark figures

Good-looking benchmark figures generated from the measured numbers (scaling curve, convergence-by-size curves, architecture ablation, text8 curve), embedded in the README and BENCHMARKS. Reproducible via scripts/plot_benchmarks.py; rendered on a white card so they read in GitHub light and dark themes. Docs/assets/tooling only.

v0.9.1 — Scaling study & a harder benchmark

A measured scaling study (bits/char vs model size, 0.6M → 10.7M — a clean diminishing-returns curve into tiny-shakespeare's data floor) and a text8 subset benchmark (1.78 bits/char, coherent out-of-domain prose), both reported honestly. Reproducible via scripts/scaling_study.py and scripts/download_text8.py.

Also backfills design docs for phases 8 & 9 and ADR-0009 (modern architecture), and hardens CI against PyTorch mirror outages. Benchmarks/scripts/docs/CI only — no library API change.

v0.9.0 — Modern architecture (RoPE, RMSNorm, SwiGLU)

The decoder is now Llama-style from scratch — RoPE, RMSNorm, SwiGLU (configurable; GPT-2-style still available).

On tiny-shakespeare it reaches 2.08 bits/char (vs 2.11 GPT-2-style) at fewer params and ~2× faster convergence — see BENCHMARKS.md for the architecture ablation. RoPE is verified correct under the KV-cache; pre-v0.9 checkpoints still load.