yes: accept invalid-Unicode arguments on Windows (#12236)#2
Open
crutkas wants to merge 1 commit into
Open
Conversation
`yes` ran a preflight `str::from_utf8` check on non-Unix platforms and rejected any argument that was not valid UTF-8, so on Windows an argument containing an unpaired UTF-16 surrogate failed with exit code 1. The output path only ever writes raw bytes (`write_all`), so the validation was unnecessary. Remove it so Windows streams the argument bytes just like Unix already does, resolving the existing `todo: accept invalid utf8`. Add a Windows regression test mirroring the Unix `test_non_utf8`. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes uutils#12236.
yesrejected non-UTF-8 arguments on Windows: a preflightstr::from_utf8check (#[cfg(not(any(unix, target_os = "wasi")))]) returned exit code 1 for any argument containing, e.g., an unpaired UTF-16 surrogate. The output path only writes raw bytes, so the check was unnecessary — removing it makes Windows stream the argument bytes exactly like Unix already does (and resolves the in-codetodo: accept invalid utf8).Test: adds
test_non_unicode_windows, the Windows analogue of the existing Unix-onlytest_non_utf8, asserting an unpaired surrogate (0xD800) is streamed as its WTF-8 bytesED A0 80instead of erroring.Verified on Windows:
cargo test -p coreutils --no-default-features --features yes --test tests -- test_yes→ 8 passed.