Skip to content

fix: always set testrun outcome to flaky_failure in process flakes #286

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

joseph-sentry
Copy link
Contributor

we were only setting this outcome to flaky failure if we were creating a new flake, but any failure on the main branch should be considered a flaky failure (by our heuristic) so to be correct we should be setting the outcome to flaky no matter if it 's for an existing flake or a new flake

a scenario where this would have been broken is if we had 2 uploads in quick succession and the flake weren't created by the first upload by the time we were processing the tests in the second, in this case we'd be missing a flaky test outcome on the second testrun

@codecov-notifications
Copy link

codecov-notifications bot commented Jul 4, 2025

Codecov Report

Attention: Patch coverage is 66.66667% with 10 lines in your changes missing coverage. Please review.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
.../shared/shared/django_apps/ta_timeseries/models.py 60.00% 6 Missing ⚠️
libs/shared/shared/django_apps/db.py 55.55% 3 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

Copy link

codecov bot commented Jul 4, 2025

Codecov Report

Attention: Patch coverage is 66.66667% with 10 lines in your changes missing coverage. Please review.

Project coverage is 94.32%. Comparing base (426c878) to head (dde7ce0).
Report is 1 commits behind head on main.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
.../shared/shared/django_apps/ta_timeseries/models.py 60.00% 6 Missing ⚠️
libs/shared/shared/django_apps/db.py 55.55% 3 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #286      +/-   ##
==========================================
- Coverage   94.35%   94.32%   -0.03%     
==========================================
  Files        1223     1224       +1     
  Lines       45310    45329      +19     
  Branches     1441     1442       +1     
==========================================
+ Hits        42751    42758       +7     
- Misses       2258     2269      +11     
- Partials      301      302       +1     
Flag Coverage Δ
apiunit 96.49% <ø> (ø)
sharedintegration 40.24% <50.00%> (+0.02%) ⬆️
sharedunit 88.60% <64.28%> (-0.06%) ⬇️
workerintegration 61.60% <0.00%> (-0.03%) ⬇️
workerunit 90.60% <100.00%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

codspeed-hq bot commented Jul 4, 2025

CodSpeed Performance Report

Merging #286 will not alter performance

Comparing joseph/fix-proc-flakes (dde7ce0) with main (a9d4685)

Summary

✅ 9 untouched benchmarks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant