SeTok

Towards Semantic Equivalence of Tokenization in Multimodal LLM

This is the repository that contains the source code for the SeTok project page.

If you use SeTok in your project, please kindly cite:

@articles{Wu2024setok,
  title={Towards Semantic Equivalence of Tokenization in Multimodal LLM},
  author={Shengqiong Wu, Hao Fei, Hanwang Zhang, Tat-Seng Chua, Shuicheng Yan},
  journal = {CoRR},
  year={2024}
}

Website License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.