v0.1.7

chonknick released this 23 Mar 04:30

· 27 commits to main since this release

a889206

What's Changed

Replace HuggingFace tokenizers with tokie for reranking
- ~20-50x faster tokenizer initialization (pre-built .tkz from CDN vs JSON parsing)
- ~260 fewer transitive dependencies
- 100% token-accurate, identical reranking results
- No runtime dependency on HuggingFace Hub
Tokenizer served from cdn.mandex.dev/models/tokenizer.tkz alongside reranker model

Assets 9