-
Notifications
You must be signed in to change notification settings - Fork 925
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Windows Runner Drops CTest Failure Output #1206
Comments
@jgehrig I saw you are using an actions The log shows:
Since they all running different commands, I am not convinced that the runner is losing certain output. |
I've done a Windows run without that plugin, and it has no effect. I've also done runs with Let me know what configuration you'd like to see across all three runs, and I will quickly switch them over. The only answer I can come up with is output is being dropped by runner. I have RDP-ed into the machine, and run both the When the same commands are run through Actions, output is missing. It looks like the missing output is from the spawned test executables (tst_input_common). |
The runner launch |
The output that appears is from Here is an example run on my local machine:
I can do the same manually on Windows while RDPed into the GH Action runner. The correct (full) output appears. |
Let me know if there is anything I can do to help debug. |
Can you point me in the direction of documentation that would describe how to debug an issue like this? I am reasonably confident the output is present, but not getting passed through to the GH Actions Runner logs... Thanks! |
Still looking for a solution here. I am not sure what is causing the output to drop from the Actions logs. |
Have this issue with a number of my repositories as well. |
I can confirm this issue for me as well |
Ping @TingluoHuang A few users have expressed they also experience this issues. Can someone take a look? I'm happy to help in any way:
Let me know if I can help. Thanks! |
The CTest output is dropped by the GitHub Actions Runner. This is a minimal repro case for demonstration. See: actions/runner#1206
Here is a minimal example of the issue: How can we debug this issue? What can we do to get ctest failure output? Let me know how I can help. |
I can confirm this, Ubuntu image works correctly, but The The problem will be somewhere between how the My current workaround for this is the following snippet:
And then invoke the |
Still an issue for me as well with the 2022 runner image and Seems this was an issue for CLion in the past so I suppose there might be something slightly unusual about how EDIT This issues seems to have nothing to do with CTest nor the arguments passed to it. Updating one of my workflow jobs to directly run a failing test executable like so (echo used as a sanity check)
Simply results in:
The entirety of the test executable's output is dropped : / |
I'm seeing this issue too. See https://github.com/louis-langholtz/PlayRho/actions/runs/5426258835/jobs/9868070006?pr=483 for an example. Search for "Failed". I'd even added the Making matters worse, I cannot get the same tests to fail on my local Windows system but suspect that the failures indicate a real issue. As a potentially helpful data point, this seems like a new problem - at least that I'm seeing for the project - in the sense that perhaps something changed on the runner or workflow environment within the last few months. I think I recall prior to that time, with different pull requests, seeing unit test failures on the Windows Debug runs that had the expected details. |
@louis-langholtz I was exactly in the same situation a few months ago, on localhost ok but on GitHub Action was failing. The big problem was that there was no way for me to investigate why it was failing because of this bug. This problem should be fixed because it is causing bad headaches 😵💫 For me the solution was the mxschmitt/action-tmate@v3, connect directly to the runner, invoke ctest manually, and obtain the output. |
I'm gonna call the mystery of dropping failure output that I was dealing with, solved. I don't believe in my case that the issue was due to the Windows runner as much as it has to do with the default Windows C++ STL runtime environment. No idea though whether what I discovered explains anyone else's missing test failure output. In my particular case, it appears there's no output by default under Windows default Data points... Tried adding
|
We can cry here maybe forever, until it will be reported at CMake issue tracker, than maybe somebody fixes it, and this could be why this problem exists so long unfixed. I wrote it like mental, but it's early morning and I'm still in half sleep 🙂 |
I just noticed that @oblivioncth earlier reported that:
|
I don't think that GitHub Actions are doing anything special than redirecting stderr and stdout outputs, I think the problem will be on CMake's side but I'm just guessing, the problem can be anywhere between or on any side. Maybe the problem is the CMake is outputting to some other output or pipe channel than the GitHub Actions are expecting. I had a similar problem with the scoop, it was outputting to some info output (7) so after the redirection I had an empty log, but it was pwsh related. |
Restores windows workflow to a closer semblance of what it used to be. Seems there's an issue with the github runner for windows that's not passing along the ctest failure info. See: actions/runner#1206
Update: I just opened a discussion in the community forum for this at https://github.com/orgs/community/discussions/60149. I'd searched for any similar discussions, but found none. Which is odd seeming to me. I mean why might we be the only ones seeing this problem? Or why is my project only recently seeing this problem on Windows when @jgehrig seems to have first seen this in 2021? Update 2023/07/16: I've closed that discussion. I've tracked down the problem I've been seeing to the pre-installed terminate handler; I don't believe that Windows runner would itself be customizing that handler. See my other comment for explanation of what was occurring in my particular situation. |
Restores windows workflow to a closer semblance of what it used to be. Seems there's an issue with the github runner for windows that's not passing along the ctest failure info. See: actions/runner#1206
- Updates to windows-2022 runner and accompanying MSVC updates. - Adds step to archive build results. There may be an issue with windows that's not showing info like from exception messages that terminate the application. See: actions/runner#1206 A particular issue that was found under windows, were aborts generating SIGABRT that apparently were due to exceptions propogating out from a destructor without any output messages containing any of the exception text info.
- Updates to windows-2022 runner and accompanying MSVC updates. - Adds step to archive build results. There may be an issue with windows that's not showing info like from exception messages that terminate the application. See: actions/runner#1206 A particular issue that was found under windows, were aborts generating SIGABRT that apparently were due to exceptions propogating out from a destructor without any output messages containing any of the exception text info.
Describe the bug
On Windows, runner drops terminal output from the test scenarios spawned by
ctest
.Working on Actions + Linux:
https://github.com/equalsraf/neovim-qt/pull/885/checks?check_run_id=3094273077
Working on AppVeyor + Win:
https://ci.appveyor.com/project/equalsraf/neovim-qt/builds/40031037/job/ddhiyj6vtqyaclt9
No Output on Actions + Win:
https://github.com/equalsraf/neovim-qt/pull/885/checks?check_run_id=3094273084
You can RDP into the Windows runner, manually run the test and observe the correct output.
The output is being dropped by the GitHub Actions Runner.
See equalsraf/neovim-qt#884 for more details.
To Reproduce
Example scenario: equalsraf/neovim-qt#885
ctest
within the action containing failures on Linux and WindowsFAIL! : ...
FAIL!: ...
orPASS: ...
Expected behavior
GitHub actions should pipe output from
ctest
, and output from the tests it spawns.You should see output for the failing tests such as
tst_encoding
.FAIL!: ...
Runner Version and Platform
Latest Github Actions hosted runner.
Does not work with
windows-latest
.Works with
ubuntu-latest
andmacos-latest
.AppVeyor + Windows example provided to demonstrate the underlying code works on Windows.
What's not working?
Please include error messages and screenshots.
Output from the failing test cases is missing. Only for GitHub Actions hosted instances running Windows.
Linux/MacOS work fine. Other CI tools work fine.
Actions + Linux (Good):
AppVeyor + Windows (Good):
Actions + Windows (Bad):
Runner and Worker's Diagnostic Logs
Here are logs from the example above:
logs_269.zip
The text was updated successfully, but these errors were encountered: