v0.10.1

Important Changes

• Japanese word segmenter model is added (#1446).

What's Changed

• chore: remove legacy ja_knbc.json model and train_ja_knbc.yml workflow by @tushuhei in #1445
• refactor(scripts): remove dead code, unused parameters, and schema fields by @tushuhei in #1447
• feat(model): add Japanese word segmenter model trained on Unicode corpus by @tushuhei in #1446
• fix(scripts): enforce UTF-8 file encoding when reading and writing datasets and models by @hardikkaurani in #1448
• chore(deps): bump the patch group across 2 directories with 6 updates by @dependabot[bot] in #1451
• chore(deps): bump the codeql group with 4 updates by @dependabot[bot] in #1452
• chore(deps-dev): bump @vitest/browser from 5.0.4 to 5.0.5 in /javascript by @dependabot[bot] in #1455
• chore(deps): bump ossf/scorecard-action from 2.4.4 to 2.5.0 by @dependabot[bot] in #1454
• chore(deps): bump actions/setup-java from 6.0.1 to 6.1.0 by @dependabot[bot] in #1453
• feat(py): don't insert separators where the source already has one by @kwy404 in #1457
• fix(py): drop boundaries before and after existing ZWSPs in HTMLChunkResolver by @reginaldalfret in #1458
• chore(deps): bump certifi from 2026.7.22 to 2026.10.19 by @dependabot[bot] in #1459
• chore(deps): remove unused Python dev dependencies by @tushuhei in #1460
• fix(scripts): use active-Hessian gain and zero-drift balancing in LogitBoost by @tushuhei in #1456
• refactor(scripts): deduplicate train.py CLI builder, dynamic out_span, and val_* naming by @tushuhei in #1450
• refactor(scripts): extract shared feature extraction and context helpers by @tushuhei in #1449
• bump version to v0.10.1 by @tushuhei in #1464

New Contributors

• @hardikkaurani made their first contribution in #1448
• @reginaldalfret made their first contribution in #1458

Full Changelog: v0.10.0...v0.10.1