GQLSA: Grouped-Query Latent Sparse Attention
Initial release of GQLSA v1.0.0
A hardware-native attention mechanism combining latent compression, grouped-query sharing, and block-sparse attention.
Key Metrics
| Metric | Value |
|---|---|
| Complexity | Linear O(T) |
| Speed vs MHA | 3.8× faster |
| KV Cache | 16× smaller |
| Perplexity | On par with dense |
What's Included
- ✅ Core attention implementation
- ✅ Complete benchmark suite
- ✅ 11 passing tests
- ✅ Research paper (PDF)
- ✅ 7 HD benchmark figures
- ✅ Usage examples
- ✅ CC BY-NC-SA 4.0 License
Links
- Paper: https://doi.org/10.5281/zenodo.22658037
- Zenodo: https://zenodo.org/records/22658037
- Repository: https://github.com/fardinsabid/gqlsa