-
-
Notifications
You must be signed in to change notification settings - Fork 6.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tests: reformat error messages to avoid tripping MSBuild #16583
Conversation
This comment was marked as resolved.
This comment was marked as resolved.
MSBuild should allow to disable this feature. If there is a way, So, all we have is this unfortunate workaround. It doesn't cover all possible stderr error message, but should catch Hm... or, perhaps we launch the test step manually, outside of CMake? That's ugly but could work in CI. Outside CI it doesn't help when when Maybe for CI we can do it anyway, just to be 100%. edit: More downsides:
|
0e89fd1
to
b1b02f0
Compare
The CI workaround doesn't work. It's also very impractical. Backtracking.
Without CMake, the VS environment is not setup, and setting it up correctly, in sync with CMake is also impractical. |
1e8d6e1
to
e2e0585
Compare
Change the format of error messages sent to stderr from tests and test
servers. As a workaround to avoid triggering Visual Studio's MSBuild
tool's built-in regexp matcher, and making it mark builds failed for
reasons we don't want them to hard fail.
Roughly, the pattern to avoid is the word "error" (case-insensitive)
in the same line with a colon
:
.It affected GHA/windows MSVC CI jobs, causing flakiness:
Ref: https://github.com/curl/curl/actions/runs/13643149623/job/38137076210?pr=16490#step:14:3125
Ref: https://github.com/curl/curl/actions/runs/13688765792/job/38277961720?pr=16582#step:14:1717
The
IgnoreStandardErrorWarningFormat="true"
MSBuild Exec optioncontrols this behavior:
https://learn.microsoft.com/visualstudio/msbuild/exec-task#parameters
I couldn't figure out a way to apply it to CMake builds.
MSBuid pattern matching rules:
https://github.com/dotnet/msbuild/blob/353c0f3d37957cc98bfa6a76b568d70d12193fc3/src/Shared/CanonicalError.cs
https://learn.microsoft.com/visualstudio/msbuild/msbuild-diagnostic-format-for-tasks
Note: There may be further error messages output from runtests scripts,
that use this format, which are not explicitly fatal. They may need
future fixes.
Thanks-to: Dion Williams
Ref: #14854 (comment)
Ref: #14854 (reply in thread)
I could not figure out a way to tell MSBuild to stop regexp matching the log and auto-fail when seeing the word "error" in it. IMO this is an unexpected and annoying feature, esp. without ways to control this behavior. It reminds of the fight with PowerShell which did the same whenever seeing anything output to stderr.