Skip to content

fix: use stdlib's utf_8_scalar_width for length calculations - #99

Merged
johnyob merged 2 commits into
johnyob:mainfrom
WardBrian:robustify-utf8-stdlib
Aug 5, 2026
Merged

fix: use stdlib's utf_8_scalar_width for length calculations #99
johnyob merged 2 commits into
johnyob:mainfrom
WardBrian:robustify-utf8-stdlib

Conversation

@WardBrian

Copy link
Copy Markdown
Contributor

Closes #93 by backporting OCaml 5.4's Format.utf_8_scalar_width and using that instead of uutf for Utf8.length. From the stdlib's doc, this function is safe to call even when the string is improperly encoded:

Invalid byte sequences are implicitly replaced by U+FFFD since this yields a better width approximation for other ascii-based encoding scheme like ISO-8859-15.

uutf is no longer required as a dependency

@WardBrian

WardBrian commented Jul 24, 2026

Copy link
Copy Markdown
Contributor Author

Note that the width in the function name stems from it's use as the default out_width function in Format, not any idea of visual width; it still considers each unicode scalar value as taking up one column

@johnyob johnyob left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm 🚀

@johnyob
johnyob merged commit 04f9280 into johnyob:main Aug 5, 2026
7 checks passed
@WardBrian
WardBrian deleted the robustify-utf8-stdlib branch August 5, 2026 14:43
shonfeder pushed a commit to ocaml/opam-repository that referenced this pull request Aug 6, 2026
CHANGES:

- fix(renderer): adjust margin for messages near end of line ([johnyob/grace#97](johnyob/grace#97))
- fix(renderer): remove extra space in trailing labels ([johnyob/grace#98](johnyob/grace#98))
- fix(renderer): error recovery on invalid UTF-8 strings ([johnyob/grace#99](johnyob/grace#99))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ansi_renderer: make more robust to invalid unicode sources

2 participants