token-v0.0.4

What's Changed

• chore: patch publish workflow for existing package versions by @chonk-lain in #54
• feat: add docs by @chonk-lain in #55
• chore: fix docs deployment by @chonk-lain in #56
• chore: format file by @chonk-lain in #57
• chore: update docs by @chonk-lain in #58
• chore: fix docs by @chonk-lain in #59
• chore: publish stable package for token by @chonk-lain in #61
• chore: fix deprecation warning by @chonk-lain in #62
• chore: fix OIDC usage for token by @chonk-lain in #63
• chore: propagate organization change by @chonk-lain in #64

Full Changelog: core-v0.0.11...token-v0.0.4

core-v0.0.11

What's Changed

• feat: add handshakes by @chonk-lain in #50
• chore: use OIDC by @chonk-lain in #52
• chore: update publish workflow by @chonk-lain in #53

Full Changelog: core-v0.0.10...core-v0.0.11

core-v0.0.10

What's Changed

• build(core): migrate from tsc+tsc-alias to tsdown by @azizbecha in #44
• Feat: update code chunker by @chonk-lain in #47
• chore: update CI by @chonk-lain in #48

New Contributors

• @azizbecha made their first contribution in #44

Full Changelog: core-v0.0.9...core-v0.0.10

core-v0.0.9

What's Changed

• feat: add semantic chunker by @chonk-lain in #40
• feat: add CodeChunker and SentenceChunker by @chonk-lain in #41
• bump version by @chonk-lain in #42
• add benchmark by @chonk-lain in #43

Full Changelog: core-v0.0.8...core-v0.0.9

core-v0.0.8

@chonkiejs/core v0.0.8

Bug Fixes

• Fix Tokenizer encode/decode for non-BMP Unicode characters (emoji, etc.) — was using charCodeAt/fromCharCode, now uses codePointAt/fromCodePoint (#33)
• Fix RecursiveChunker sentence-level splitting being completely non-functional — multi-char delimiters like ['. ', '! ', '? '] were joined into a string where space became a delimiter, causing all segments to collapse into one (#39)
• Fix TokenChunker silent index failure — replaced fragile indexOf-based lookup with deterministic offset tracking (#35)
• Fix TableChunker startIndex/endIndex inconsistency between single and multi-chunk cases (#36)
• Consolidate duplicate WASM initialization from FastChunker and RecursiveChunker into shared module (#37)

Tests

• Add Tokenizer unit tests (9 tests) — encode/decode round-trip, Unicode, create factory
• Add TokenChunker test coverage (14 tests) — chunking, overlap, indices, Unicode (#34)
• Add TableChunker index consistency test
• Total tests: 47 → 71

core-v0.0.7

@chonkiejs/core v0.0.7

• Integrate @chonkiejs/chunk WASM for split and merge operations
• Use WASM split_offsets for delimiter-based text splitting
• Use WASM merge_splits for combining segments into chunks
• Add initWasm function for explicit WASM initialization
• Re-export low-level chunk functions from @chonkiejs/chunk

v0.0.4

What's Changed

• Refactor: Re-write chonkie-ts into separate packages for better dependency handling! by @chonknick in #27

Full Changelog: https://github.com/chonkie-inc/chonkiejs/commits/v0.0.4