Skip to content

v0.2.3 - Faster flow stepping (bit-identical)

Choose a tag to compare

@Saganaki22 Saganaki22 released this 16 Aug 05:33
· 3 commits to main since this release

Patch Release

Faster generation with provably identical output, plus repo/licensing cleanup since v0.2.0.

Performance: optimized flow stepping

Per patch, the flow DiT ran 10 euler steps and recomputed values that never change across a generation. Now:

  • Timestep embeddings, per-block AdaLN modulations (11 blocks x 10 steps) and rotary frequencies are computed once per generation instead of once per patch (~500+ kernel launches removed per patch)
  • Generation buffers are preallocated (no more growing torch.cat per patch) and the CFG null-condition is hoisted

Measured (RTX 5090, bf16 repo, flash_attention, seed-42 voice clone, 203 patches / 32.5s audio):

patches/s output
v0.2.0 2.79 reference
v0.2.3 3.23 (+16%) bit-identical (max abs diff 0.0, same stop point)

Also evaluated and deliberately rejected torch.compile on the flow DiT: 2.15x per-patch but fusion drift compounds through the autoregressive loop - in testing the stop head fired 182 patches late and produced hallucinated trailing speech. Not shippable. This release takes only the exact-math wins.

Also since v0.2.0

  • Repo license switched to MIT (model weights remain Apache-2.0 under upstream's license)
  • README: directory-tree model layout, screenshot + language links, dropped stale tools/ references
  • tools/ folder removed from the repo (quantization/validation scripts remain in git history and were used to produce the int8 mirror)
  • New tests/test_flow_opt.py equivalence tests; full suite 13/13 passing