This is the implementation of SpaceByte used in SpaceByte: Towards Deleting Tokenization from Large Language Modeling.
SpaceByte is a tokenization-free large language model (LLM) that uses multiscale modeling at the byte and word levels to model language with the same performance as standard LLM architectures that instead use tokenization.
See the reproduce directory for instructions for reproducing the results in our paper.
