fix: take flaky_fail_count into account for total tests in new TA impl #275
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
in the old TA implementation the possible outcomes were: pass, fail, and skip. flaky failures were a subset of fails which meant we shouldn't include the flaky_fail_count in the total tests calculation In the new implementation that's not the case, flakiness is represented by the outcome of a testrun, which means that we have to count flaky fails in the total tests This commit also does some light refactoring of the polars aggregation expressions for simplicity, and fixes a bug where we weren't running fill_nan when it was needed.
Codecov ReportAll modified and coverable lines are covered by tests ✅ ✅ All tests successful. No failed tests found. 📢 Thoughts on this report? Let us know! |
Codecov ReportAll modified and coverable lines are covered by tests ✅
✅ All tests successful. No failed tests found. Additional details and impacted files@@ Coverage Diff @@
## main #275 +/- ##
==========================================
+ Coverage 94.35% 94.36% +0.01%
==========================================
Files 1223 1223
Lines 45309 45312 +3
Branches 1441 1441
==========================================
+ Hits 42750 42759 +9
+ Misses 2258 2252 -6
Partials 301 301
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
joseph-sentry
requested review from
ElioDiNino,
ajay-sentry and
michelletran-codecov
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
in the old TA implementation the possible outcomes were: pass, fail, and skip.
flaky failures were a subset of fails which meant we shouldn't include the
flaky_fail_count in the total tests calculation
In the new implementation that's not the case, flakiness is represented by the
outcome of a testrun, which means that we have to count flaky fails in the total
tests
This commit also does some light refactoring of the polars aggregation
expressions for simplicity, and fixes a bug where we weren't running fill_nan
when it was needed.