Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[trainer] Fix reports failure with skipped tests #20045

Merged

Conversation

tahirmt
Copy link
Contributor

@tahirmt tahirmt commented Mar 10, 2022

Trainer would fail if there are any skipped tests in case of retries.

Checklist

  • I've run bundle exec rspec from the root directory to see all new and existing tests pass
  • I've followed the fastlane code style and run bundle exec rubocop -a to ensure the code style is valid
  • I've read the Contribution Guidelines
  • I've updated the documentation if necessary.

Motivation and Context

Noticed that when using trainer with retried tests, it would fail the action even when the end result with retries is success. Using number_of_failures_excluding_retries to fix the issue.

I think the issue is the same as this. Resolves #19687

Description

Changed the check for tests_successful for trainer test_parser` to use number of failures excluding retries

Testing Steps

Use trainer on an xcresult file that includes retries but with an overall successful result with fail_build set to tree and expect trainer to succeed.

@google-cla
Copy link

google-cla bot commented Mar 10, 2022

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

For more information, open the CLA check for this pull request.

@tahirmt tahirmt force-pushed the fix-trainer-failing-for-skipped-tests branch from 46b5f7f to bf9a1c4 Compare March 10, 2022 03:12
@tahirmt
Copy link
Contributor Author

tahirmt commented Mar 10, 2022

I'm sorry for the force push but I committed with the wrong email address and couldn't sign the CLA without it.

@tahirmt tahirmt changed the title [trainer] reports failure with skipped tests [trainer] Fix reports failure with skipped tests Mar 10, 2022
Trainer would fail if there are any skipped tests in case of retries.
@tahirmt tahirmt force-pushed the fix-trainer-failing-for-skipped-tests branch from bf9a1c4 to 90718ee Compare March 23, 2022 05:04
Copy link
Member

@joshdholtz joshdholtz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes so much sense! Thank you for catching and fixing ❤️

@joshdholtz joshdholtz merged commit 74715ce into fastlane:master Mar 24, 2022
Copy link

@fastlane-bot fastlane-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Congratulations! 🎉 This was released as part of fastlane 2.205.1 🚀

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fastlane test result doesn't align with Xcode xcresult if Xcode 13 build-in "retry on failure" is active.
3 participants