-
Notifications
You must be signed in to change notification settings - Fork 113
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add skip
as a supported custom result outcome
#2326
Conversation
@happz Rebased, I think i addressed all the issues. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the improvement. Looks good, added just two comments. And thinking: What about extending the /tests/execute/basic
test to exercise the new skip
status as well?
The final summary shows:
Perhaps we could use |
/packit test |
@mkoncek, are you going to address the pending comments? |
Yes, I have been meaning to do it for some time... Regarding documentation, the wording will need to be very precise. |
/packit test |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There should be a test for this outcome as well.
/packit test |
Some changes will be needed also on the Testing Farm side: TFT-2282 |
/packit test |
The use case is when a single tmt test produces multiple results but some of them were not executed because the process decided so. It should be visible to the user that such a test exists and that there is not a bug in the implementation which caused the test to be missing. Co-authored-by: Petr Šplíchal <psplicha@redhat.com>
/packit test |
skip
custom result outcomeskip
as a supported custom result outcome
As we discussed with @lukaszachy probably a month ago the purpose of this is to explicitly state that a test was not executed.
The use case is when a single tmt test produces multiple results but some of them were not executed because the implementation decided so. It should be visible to the user that such a test exists and that there is not a bug in the implementation which caused the test to be missing.
It is still an open question how such cases should be displayed in Fedora's CI.