Route verbose test output to file to reduce CI log size - #54832
Route verbose test output to file to reduce CI log size#54832marcpopMSFT wants to merge 1 commit into
Conversation
Add TestCommand.SuppressOutputOnFailure property that writes command output to a file on disk (in HELIX_WORKITEM_UPLOAD_ROOT or temp) instead of dumping it to the test log when a command fails. The output is still captured in CommandResult.StdOut/StdErr for assertions. Apply this to ItUsesVerbosityPassedToDefineVerbosityOfConsoleLoggerOfTheTests for the 'd' and 'diag' verbosity cases, which intentionally produce ~95,000 lines of MSBuild diagnostic output per invocation. This saves ~19 MB per Helix shard that runs these tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Reduces CI/Helix log size by redirecting intentionally-verbose failing command output to a file (preferably in the Helix upload directory) while still keeping stdout/stderr available for assertions.
Changes:
- Add
TestCommand.SuppressOutputOnFailureto write command output to disk on failure instead of dumping it into the test log. - Emit a single-line pointer to the saved output file (with basic stdout/stderr line counts).
- Enable suppression for the
-v dand-v diagcases in thedotnet testverbosity behavior test.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| test/Microsoft.NET.TestFramework/Commands/TestCommand.cs | Adds an opt-in mechanism to persist large failing output to disk instead of logging it inline. |
| test/dotnet.Tests/CommandTests/Test/GivenDotnetTestBuildsAndRunsTestfromCsproj.cs | Opts the high-verbosity failure cases into output suppression to cut CI log volume. |
| // Write output to a file instead of the test log to avoid bloating CI logs. | ||
| var outputDir = Environment.GetEnvironmentVariable("HELIX_WORKITEM_UPLOAD_ROOT") | ||
| ?? Path.GetTempPath(); | ||
| var fileName = $"cmd-output-{Guid.NewGuid():N}.log"; | ||
| var outputPath = Path.Combine(outputDir, fileName); | ||
| File.WriteAllText(outputPath, | ||
| $"> {result.StartInfo.FileName} {result.StartInfo.Arguments}{Environment.NewLine}" + | ||
| $"Exit code: {result.ExitCode}{Environment.NewLine}{Environment.NewLine}" + | ||
| $"=== STDOUT ==={Environment.NewLine}{result.StdOut ?? ""}{Environment.NewLine}{Environment.NewLine}" + | ||
| $"=== STDERR ==={Environment.NewLine}{result.StdErr ?? ""}"); | ||
| int stdOutLines = string.IsNullOrEmpty(result.StdOut) ? 0 : result.StdOut.Split(new[] { Environment.NewLine }, StringSplitOptions.None).Length; | ||
| int stdErrLines = string.IsNullOrEmpty(result.StdErr) ? 0 : result.StdErr.Split(new[] { Environment.NewLine }, StringSplitOptions.None).Length; | ||
| Log.WriteLine($" ⚠️ Command failed — output ({stdOutLines} stdout + {stdErrLines} stderr lines) written to: {outputPath}"); |
|
@MichaelSimons as a follow up to my other PR, copilot identified an additional test that was leading to large output. After going back and forth, this was a potential fix we came up with where if it fails, the output will end up in a file rather than sent to the test output. I'm not sure if this will end up with similar problems to what you reported with dmp copying if this pattern ends up in other tests as well. Copilot also suggested we could cap the output at 500 lines but I worried that then we wouldn't be able to investigate as easily so I didn't include that. |
| /// The output is still captured in <see cref="CommandResult.StdOut"/>/<see cref="CommandResult.StdErr"/> | ||
| /// for assertions. | ||
| /// </summary> | ||
| public bool SuppressOutputOnFailure { get; set; } |
There was a problem hiding this comment.
The name SuppressOutputOnFailure suggests the output is discarded. I think will hurt the readability of the code and prevent some from using it when they don't read the full doc. Since the output is actually redirected to a file (and still available for download), consider a name like RedirectOutputToFileOnFailure that conveys the output is preserved, not lost.
Summary
Follow-up to #54782. Addresses the remaining ~10 MB per shard caused by tests that intentionally invoke MSBuild with diagnostic verbosity.
Problem
After #54782, three dotnet.Tests.dll shards remained at ~9.5-9.8 MB each. Root cause: ItUsesVerbosityPassedToDefineVerbosityOfConsoleLoggerOfTheTests runs dotnet test -v diag, producing ~95,000 lines of MSBuild output. Since the test expects exit code 1 (intentional failure), the dump-on-failure logic writes all of it to the test log.
Solution
Added TestCommand.SuppressOutputOnFailure property:
Applied to the verbosity d and diag cases of the problematic test.
Expected impact
~19 MB less per PR build run across the affected shards (from ~29 MB to ~10 MB for those 3 shards combined).
Debugging
If you need the full output locally, set DOTNET_SDK_TEST_VERBOSE=1 to stream everything in real-time, or check the output file on disk / in Helix uploads.