Skip to content

v0.5.2: truncate_utf8 non-iconv fallback now drops invalid UTF-8 anywhere in the text (#77)

Choose a tag to compare

View all tags
@github-actions github-actions released this 08 Sep 01:49
· 11 commits to main since this release
v0.5.2
7c2140d
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

v0.5.2: truncate_utf8 non-iconv fallback now drops invalid UTF-8 anywhere in the text (#77)

Bug fix. When iconv is unavailable, truncate_utf8 previously trimmed only an
incomplete trailing sequence, so a stray invalid byte inside --text survived
into an emitted ledger row. The fallback is now a forward byte walk that keeps
the longest RFC 3629 well-formed prefix (the same result the iconv branch
computes). Test-only seam SITTER_TEST_NO_ICONV=1; two new tests; docs state
the guarantee per validator (macOS libiconv is lenient on forms above
U+10FFFF and obsolete 5/6-byte sequences).

Evidence: PR #79 (L1-7 record; CI on
8e8e66b green on ubuntu and macOS; seats Opus 5 / Kimi K3 / Qwen 3.8 Max r3 GO x3).
Follow-up: #80

Assets 2
Loading