Repository navigation

v0.4.0

qustrolabe released this 10 Jan 00:26

· 9 commits to main since this release

f8c076a

* some performance improvements

* add no tokens mode, rethink token count strategy

* fix README.md typo

* added tests, markdown decorator, few tokenization performance improvements

* pedantic clippy fix

* version increment

Assets 6