Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

appveyor: disable test 571 in two cmake builds #6119

Closed
wants to merge 1 commit into from

Conversation

@bagder
Copy link
Member

@bagder bagder commented Oct 22, 2020

... they're simply too flaky there.

...  they're simply too flaky there.
@bagder bagder added the CI label Oct 22, 2020
@MarcelRaad
Copy link
Member

@MarcelRaad MarcelRaad commented Oct 22, 2020

Why not mark them as flaky? Is there anything special to these two build configurations?

@bagder
Copy link
Member Author

@bagder bagder commented Oct 22, 2020

Only because I don't want to disable them for every build everywhere.

Ideally we introduce a way in the future to give us more qualifiers to "flaky". Like platforms, build combinations etc.

@dfandrich
Copy link
Collaborator

@dfandrich dfandrich commented Oct 22, 2020

@mback2k
Copy link
Member

@mback2k mback2k commented Oct 22, 2020

For this I introduced the ~ prefix for test numbers in runtests.pl. It does not disable them, but ignores the result in the CI status.

@mback2k
Copy link
Member

@mback2k mback2k commented Oct 22, 2020

Also this seems to be the AppVeyor equivalent of #5954. Unfortunately I wasn't able to debug this RTSP test on Windows (yet).

appveyor.yml Show resolved Hide resolved
@bagder
Copy link
Member Author

@bagder bagder commented Oct 24, 2020

Just mark this test as flaky-cmake

But its cmake-on-appveyor, or possibly cmake-on-win32. I think I rather stick to disabling it in appveyor for now.

@dfandrich
Copy link
Collaborator

@dfandrich dfandrich commented Oct 24, 2020

@bagder bagder closed this in 6f88718 Oct 24, 2020
@bagder bagder deleted the bagder/appveyor-test571 branch Oct 24, 2020
@mback2k
Copy link
Member

@mback2k mback2k commented Oct 25, 2020

Actually it is either not working due to MinGW, MSYS or Win32 since it is also broken for the autotools based builds on Azure Pipelines. So I don't think this is caused by CMake.

@bagder
Copy link
Member Author

@bagder bagder commented Oct 25, 2020

Short term, we can keep adding test numbers to ignore in the CI setup files, and then we can go over them all and see which are ignored where and see if we can draw conclusions and/or make better systems to handle them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants
You can’t perform that action at this time.