Skip to content

fmt: measure -w in UTF-8 bytes to match GNU - #13651

Merged
RenjiSann merged 1 commit into
uutils:mainfrom
sankalpsthakur:fix/fmt-gnu-width-10095
Jul 30, 2026
Merged

fmt: measure -w in UTF-8 bytes to match GNU#13651
RenjiSann merged 1 commit into
uutils:mainfrom
sankalpsthakur:fix/fmt-gnu-width-10095

Conversation

@sankalpsthakur

Copy link
Copy Markdown
Contributor

Summary

GNU fmt (verified against 9.11) measures -w/--width in UTF-8 bytes, not Unicode display columns. uutils was using unicode-width, so CJK examples from #10095 wrapped differently.

This switches char_width / word length accounting in parasplit.rs to UTF-8 byte length and adds regression tests for the issue’s multibyte cases.

Intentionally not reopening #12623’s width-1 / newline-budget approach (closed as stale vs GNU 9.10+).

When GNU grows true multibyte/display-column width support, this should be revisited.

Validation

cargo test --features unix --test tests test_fmt_width -- --nocapture
# 7 passed (includes test_fmt_width_multibyte_gnu_compatible)

Compared local binary output to GNU 9.11 on the #10095 examples.

Fixes #10095

AI/LLM disclosure

  • AI coding tools (including Grok and/or Codex agent-assisted editing) were used to help draft or modify code and this PR description.
  • I reviewed the complete change, understand the reasoning, and ran the reported local tests before submitting.
  • This submission is original work of authorship under the project contributor terms; AI output was not pasted unreviewed.

Made with Cursor

GNU fmt (through 9.11) is not multibyte-aware and counts line length
from pointer differences, not Unicode display columns. Switch uutils
fmt to the same byte-oriented width so CJK wrapping matches GNU.
Revisit when GNU moves to display-column width.

Fixes uutils#10095

Co-authored-by: Cursor <cursoragent@cursor.com>
@codspeed-hq

codspeed-hq Bot commented Jul 30, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 4.28%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

❌ 2 regressed benchmarks
✅ 343 untouched benchmarks
⏩ 46 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation du_summarize_balanced_tree[(5, 4, 10)] 15.7 ms 16.5 ms -4.3%
Simulation du_max_depth_balanced_tree[(6, 4, 10)] 60.9 ms 63.6 ms -4.27%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing sankalpsthakur:fix/fmt-gnu-width-10095 (875fac9) with main (0cb3473)

Open in CodSpeed

Footnotes

  1. 46 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@github-actions

Copy link
Copy Markdown

GNU testsuite comparison:

Skip an intermittent issue tests/date/resolution (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tail/tail-n0f (fails in this run but passes in the 'main' branch)
Note: The gnu test tests/tail/pipe-f is now being skipped but was previously passing.

@RenjiSann

Copy link
Copy Markdown
Collaborator

Thanks !

@RenjiSann
RenjiSann merged commit 93025b5 into uutils:main Jul 30, 2026
164 of 168 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fmt: line-wrapping differs from gnu implementation in some situations

2 participants