At the moment this is a super pre-mature release of this codebase, but felt that the code here is useful imo.
Working ATM:
- My own implementation of hash-Moe: https://arxiv.org/pdf/2106.04426.pdf
- Various Iterable Loaders and prepare dataset scripts
- My own implementation of MoD: https://arxiv.org/abs/2404.02258
- Transformer, RNN, and Convolutional Components
- An Encoder Transformer Useful for Classification
- other bits and bobs