possible papers (and code) of interest

[Thomas-MMJ]

Have you had a look at bitsandbytes?

https://github.com/TimDettmers/bitsandbytes

https://arxiv.org/abs/2208.07339

https://timdettmers.com/2022/08/17/llm-int8-and-emergent-features/

Also this paper on tradeoffs for various 8 bit quantization formats,

https://arxiv.org/pdf/2206.02915v1.pdf

[lucidrains]

@Thomas-MMJ hmm, that is something entirely different

[Thomas-MMJ]

@Thomas-MMJ hmm, that is something entirely different

Ok wasn't sure if either of them were relevant.