Skip to content

v1.4.1

Choose a tag to compare

@amirivojdan amirivojdan released this 22 Feb 03:00
· 62 commits to main since this release

What’s New in v1.4.1

This release focuses on bug fixes and refactoring for more robust Persian text processing in Shekar.

Highlights

  • Fixed Sentence Tokenizer Bug – handles Persian quotations and multi-sentence structures correctly.
  • Fixed Spellchecker Bug – improved accuracy and stability.
  • Refactored Spacing Normalizer – split into four well-defined modules:
    • OrthographicSpacingNormalizer
    • PunctuationSpacingNormalizer
    • WordSpacingNormalizer
    • VerbalSpacingNormalizer
      Enhances clarity, maintainability, and modularity of preprocessing pipelines.