v10.2.1

Fixes

fix: support soft hyphens in the word splitter (#9144)

fix: support soft hyphens in the word splitter (#9144)

This pull request introduces several improvements and fixes to the word segmentation and word splitting logic, especially for Thai language support and symbol handling. The main changes include updating the Thai segmentation to use soft hyphens, enhancing the word splitting logic to handle more cases (including camelCase, symbols, and soft hyphens), and refactoring the code for better modularity and test coverage.

Word segmentation and Thai language improvements:

• Updated Thai (th-TH) segmentation to use soft hyphens (\u00AD) instead of spaces, improving the accuracy of word breaks and ensuring better compatibility with spell checking and downstream processing. [1] [2] [3]
• Added the @cspell/dict-th-th package to dependencies, enabling Thai dictionary support.
• Adjusted tests and expectations for Thai segmentation to reflect the use of soft hyphens and to ensure no false positives in validation.

Word splitting and symbol handling enhancements:

• Introduced a new generateWordBreaks utility that generates all possible word breaks in a string, including camelCase, symbols, numbers, and soft hyphens. Includes comprehensive tests and snapshot coverage for various edge cases. [1] [2] [3] [4]
• Improved regular expressions for word splitting to support soft hyphens and better handle punctuation and symbols.

Refactoring and code organization:

• Refactored imports to use the new wordSplitter/index.js entry point, improving modularity and future maintainability. [1] [2]
• Added explicit exports for split, SplitOptions, and SplitResult in wordSplitter/index.ts to clarify and centralize the public API.

Other enhancements:

• Exposed the softHyphen constant for consistent use throughout the codebase. [1] [2]

---

Dictionary Updates

fix: Workflow Bot -- Update Dictionaries (main) (#9143)

fix: Workflow Bot -- Update Dictionaries (main) (#9143)

Update Dictionaries (main)

Summary

 packages/cspell-bundled-dicts/package.json |  2 +-
 pnpm-lock.yaml                             | 25 +++++++++----------------
 2 files changed, 10 insertions(+), 17 deletions(-)

---