Files

 main

/

tokenizer

/

Copy path

Directory actions

More options

Directory actions

More options

Latest commit

kemingy

feat: add more tokenizers (#28)

Jan 8, 2025

1463c7a · Jan 8, 2025

History

History

/

tokenizer

/

Top

Folders and files

Name		Name	Last commit message	Last commit date
parent directory ..
bert_base_uncased.json		bert_base_uncased.json	feat: init with basic index build and scan	Sep 26, 2024
wiki_tocken.json		wiki_tocken.json	feat: add more tokenizers (#28)	Jan 8, 2025
View all files