Skip to content

v0.6.0 - KD loss vectorized (~16x), ADR-0019 OPST

Choose a tag to compare

View all tags
@log0u7 log0u7 released this 28 Oct 19:59
· 176 commits to main since this release
v0.6.0
349841b
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Retroactive cut of the Oct 28 morning wave.

Training

  • KD loss fully vectorized: first optimizer step went from 28+ minutes (issue #57) to ~1.7 min/step - a ~16x real-world speedup on the gpu3080 workload. Vocabulary cached once, targets as tensors, batched loss (#63).
  • Tokenizer vocabulary cache in the KD loop (#58).

Docs / decisions

  • ADR-0019: on-policy self-training (OPST) accepted (#62).
  • Benchmark documentation v1: perplexity table + runbook; README moves to 1.0-rc status (#59-#61).
Assets 2
Loading