Skip to content

v0.1.2

Choose a tag to compare

View all tags
@briney briney released this 10 Jun 16:14
· 21 commits to main since this release
v0.1.2
d7c99c1
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's Changed

  • Add architecture ablation toggles: mask dropout, L2 QKNorm, residual gates, GEGLU, gated attention by @briney in #7
  • Implement ResFormer value residual connections by @briney in #8

Full Changelog: v0.1.1...v0.1.2

Contributors

briney
Assets 2
Loading