[trainer] Fix reports failure with skipped tests #20045
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Trainer would fail if there are any skipped tests in case of retries.
Checklist
bundle exec rspec
from the root directory to see all new and existing tests passbundle exec rubocop -a
to ensure the code style is validMotivation and Context
Noticed that when using
trainer
with retried tests, it would fail the action even when the end result with retries is success. Usingnumber_of_failures_excluding_retries
to fix the issue.I think the issue is the same as this. Resolves #19687
Description
Changed the check for
tests_successful
fortrainer
test_parser` to use number of failures excluding retriesTesting Steps
Use trainer on an xcresult file that includes retries but with an overall successful result with
fail_build
set totree
and expect trainer to succeed.