Skip to content

TextHumanize 0.31.1 — minimal-mode whitespace fix

Choose a tag to compare

@ksanyok ksanyok released this 11 Jun 09:12
· 5 commits to main since this release

Fixed

  • Whitespace corruption in minimal=True / only_flagged=True. Selective humanization dropped the space after every sentence-ending period — "Sample size looked right. Geographic spread looked right." became "...right.Geographic...". _humanize_flagged_only rejoined detect_ai_sentences() output by its trimmed text while the surrounding whitespace lived inside [start:end] (so end == next start and the gap-recovery branch never fired). The fix reattaches both leading and trailing whitespace of each sentence span: clean input now round-trips byte-for-byte, and rewrites keep their spacing. This also restores meaningful internal AI-score deltas for these modes (previously inflated because the missing spaces tanked burstiness/perplexity).

Reported by a client via @ksanyok. Regression tests in tests/test_minimal_whitespace.py.

No change to the default humanize() path, batch, or chunked modes.