Ignore non-utf8 text in build script output #1583
Conversation
| if let Some(bso) = BuildScriptOutput::new(&line) { | ||
| let mut buf = Vec::new(); | ||
| while reader | ||
| .read_until(b'\n', &mut buf) |
There was a problem hiding this comment.
Could you add a test that if the output doesn't end with a trailing newline, we still parse the last line of output? (i.e. can you remove the trailing newline from the example output in the test?)
Also, could you add a line of invalid utf-8 to the testdata in the test, to verify this change?
| .expect("Cannot read line") | ||
| != 0 | ||
| { | ||
| let line = &String::from_utf8_lossy(&buf); |
There was a problem hiding this comment.
It looks like cargo actually completely skips any line which isn't utf-8: https://github.com/rust-lang/cargo/blob/0b84a35c2c7d70df4875a03eb19084b0e7a543ef/src/cargo/core/compiler/custom_build.rs#L566-L569
Can you do the same, rather than trying to parse the line?
|
Sorry, didn't notice the failed clippy test. |
|
Sorry, it looks like you need to merge main and then we can get this merged! |
When compiling the latest zstd crate on a German locale in non-release mode on Windows, it outputs localized messages that are not UTF8, causing the compile to fail.
|
(rebased on main) |
When compiling the latest zstd crate on a German locale in non-release mode on Windows, it outputs localized messages that are not UTF8, causing the compile to fail.