Skip to content

ANEForge 0.1.3

Choose a tag to compare

@sbryngelson sbryngelson released this 22 Jun 16:35
· 324 commits to main since this release
a549bc7

Query-tiling for the attention helpers.

  • af.sdpa and af.mha tile the query at large sequence, materializing [tile, Skv] score blocks instead of the full score matrix (~3x at S=1500). Exact (#17).
  • cross_attention tiles the query when both query and context are long (S >= 768 and T >= 512, ~2.4x past the materialization wall). Small-T cross-attention (e.g. SD text conditioning) stays single-shot and byte-identical (#19).

No API changes.