Release v1.0.7 · feyninc/chonkie

✨ Highlights

Added initial support for Handshakes and Porters with 1 new Porter (JSONPorter) as well as 3 new Handshakes (ChromaHandshake, QdrantHandshake and TurbopufferHandshake)! Extemely simple to use, you can just import, init and call on your List[Chunks] that you get out from your chunkers/refineries and query them for search!
Added support for OpenAIGenie which allows you to use the OpenAI models with the SlumberChunker and many more! Any API that supports the OpenAI API format will be usable with the base_url changed. Try out various LLMs with the SlumberChunker to see which works best for you!
Added support for VoyageAIEmbeddings (thanks to @Pratik960): Use VoyageAI models with EmbeddingsRefinery and SemanticChunkers for your ingestion~
Added new themes to the Visualizer: Dark mode and a retro tiktokenizer theme (thanks to @Udayk02)
Fixes and perfomance improvements in the NeuralChunker
New one-page markdown DOCS.md for easy answers about Chonkie with LLMs.

Feat: Add tiktokenizer theme to the Visualizer by @chonknick in #81
Feat: Added support for VoyageAI embeddings by @Pratik960 in #67
Feat: Add initial support for VoyageAIEmbeddings by @chonknick in #85
Fix: Enhance NeuralChunker by adding support for multiple models, custom… by @chonknick in #86
Add the local DOCS.md for consistency and ease by @chonknick in #89
Update DOCS.md by @chonknick in #90
Add BaseHandshake and BasePorter by @chonknick in #91
Update cookbook by @chonknick in #92
Feat: Add ChromaHandshake + QdrantHandshake + Document + Chomp + More by @chonknick in #93
Feat: Add support for OpenAIGenie + TurbopufferHandshake by @chonknick in #95
Update README.md by @chonknick in #96
Add warning for experimental status of TurbopufferHandshake in turbop… by @chonknick in #97
Feat: Viz: Added a new dark theme, separated the light and the dark themes by @Udayk02 in #82
Feat: Add dark mode to chonkie's Visualizer by @chonknick in #98
Update DOCS.md by @chonknick in #99

Full Changelog: v1.0.6...v1.0.7