v1.5.4

Chonkie v1.5.4 🎉

Chonkie CLI 💻

checkout the docs for more details about CLI commands and usage

Migrate from Cython to rust ⚡

we have officially migrated from Cython towards rust 🦀

---

What's Changed

• feat: add chonkie CLI by @chonk-lain in #401
• Register Milvus handshake by @akx in #436
• Use component registry in CLI by @akx in #435
• CI: speed up & simplify by @akx in #420
• perf: defer Hubbie recipe schema download and parsing by @akx in #428
• Fix FastChunker UTF-8 byte offset mismatch by @chonknick in #440
• feat: Add text extraction mode to SlumberChunker by @chonknick in #443
• fix: Use group_end_index as fallback when extraction fails by @chonknick in #444
• fix: enable Ruff B904; use raise ... from ... when reraising by @akx in #424
• chore(deps): bump aiohttp from 3.13.0 to 3.13.3 by @dependabot[bot] in #448
• chore(deps): bump pyasn1 from 0.6.1 to 0.6.2 by @dependabot[bot] in #449
• chore(deps): bump langchain-core from 0.3.78 to 0.3.81 by @dependabot[bot] in #450
• chore(deps): bump urllib3 from 2.5.0 to 2.6.3 by @dependabot[bot] in #446
• chore(deps): bump authlib from 1.6.5 to 1.6.6 by @dependabot[bot] in #447
• Bump azure-core from 1.35.1 to 1.38.0 by @dependabot[bot] in #442
• Bump filelock from 3.19.1 to 3.20.3 by @dependabot[bot] in #441
• feat: Add GroqGenie and CerebrasGenie for fast inference by @chonknick in #453
• docs: Add GroqGenie and CerebrasGenie documentation by @chonknick in #454
• refactor: replace Cython split/merge with chonkie-core (Rust) by @chonknick in #456
• refactor: replace memchunk with chonkie-core in FastChunker by @chonknick in #457
• feat: Register official SentenceTransformer models to resolve TODO by @chimchim89 in #445
• Format chunker modules with ruff by @chimchim89 in #467
• add lain to assignees list by @chonk-lain in #468
• update cli by @chonk-lain in #439
• fix: add exception chaining to CLI error handling by @chonknick in #470
• Add tenacity retry logic to OpenAI API calls by @emerose in #198
• feat: update publish workflow by @chonk-lain in #474
• feat: remove api docs by @chonk-lain in #475
• bump version by @chonk-lain in #477

New Contributors

• @chimchim89 made their first contribution in #445
• @emerose made their first contribution in #198

Full Changelog: v1.5.2...v1.5.4