Skip to content

ref(ci-st): fast calculate-shards heuristics#112319

Merged
joshuarli merged 3 commits into
masterfrom
ref-ci-st-fast-calculate-shards
Apr 7, 2026
Merged

ref(ci-st): fast calculate-shards heuristics#112319
joshuarli merged 3 commits into
masterfrom
ref-ci-st-fast-calculate-shards

Conversation

@joshuarli
Copy link
Copy Markdown
Member

no more setup-sentry and pytest --collect-only! test count is only used to calculate shards and we don't need 100% accuracy, even ~90% is acceptable.

this completely eliminates 2m serial per-workflow!

we now count test_ with a regex across SELECTED_FILES and ast-based parsing for @pytest.mark.parametrize multipliers (inline lists, variable references, subscript access, stacked decorators, stored decorators).

this is ~99% accurate because there are only 7 @pytest.fixture(params=...) which cannot be statically determined, and we have ~30k tests

@joshuarli joshuarli requested a review from a team as a code owner April 6, 2026 23:29
@joshuarli joshuarli requested a review from rbro112 April 6, 2026 23:29
@github-actions github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label Apr 6, 2026
Comment thread .github/workflows/scripts/calculate-backend-test-shards.py Outdated
Comment thread .github/workflows/scripts/calculate-backend-test-shards.py Outdated
Comment thread .github/workflows/scripts/calculate-backend-test-shards.py
Copy link
Copy Markdown
Contributor

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 525fa4f. Configure here.

Comment thread .github/workflows/scripts/test_calculate_backend_test_shards.py
@joshuarli joshuarli merged commit 7585e12 into master Apr 7, 2026
59 of 60 checks passed
@joshuarli joshuarli deleted the ref-ci-st-fast-calculate-shards branch April 7, 2026 00:10
george-sentry pushed a commit that referenced this pull request Apr 9, 2026
no more setup-sentry and pytest --collect-only! test count is only used
to calculate shards and we don't need 100% accuracy, even ~90% is
acceptable.

this completely eliminates 2m serial per-workflow!

we now count test_ with a regex across SELECTED_FILES and ast-based
parsing for `@pytest.mark.parametrize multipliers` (inline lists,
variable references, subscript access, stacked decorators, stored
decorators).

this is ~99% accurate because there are only 7
`@pytest.fixture(params=...)` which cannot be statically determined, and
we have ~30k tests
@github-actions github-actions Bot locked and limited conversation to collaborators Apr 22, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants