chore(warehouse-sources): stop slowest CI tests wasting runner time - #70291
Merged
Conversation
Two changes targeting the top of the slowest-tests report: - test_retryable_statuses_raise did real tenacity exponential-backoff sleeps (~3 min per parametrized case, 5 cases, every CI run). Patch tenacity's sleep so retries run instantly while still exercising the full retry count and reraise behavior. - Delete test_migration_0014.py: the 0014 backfill is an elidable one-off data migration with dozens of migrations shipped on top of it, and the TestMigrations base replays migrations in setUp for each of the 10 expanded test IDs. Generated-By: PostHog Code Task-Id: 55a3d6ea-3391-4b9e-91fb-a2da870c2b05
Contributor
|
Hey @pauldambra! 👋 It looks like your git author email on this PR isn't your
You can fix it for this repo with: git config user.email "you@posthog.com"Or set it globally with |
pauldambra
marked this pull request as ready for review
July 12, 2026 13:20
pauldambra
enabled auto-merge (squash)
July 12, 2026 13:21
![stamphog[bot]](https://avatars.githubusercontent.com/in/3000332?s=60&v=4){kind=small}
There was a problem hiding this comment.
Test-only change: mocks tenacity's sleep so a slow retry test runs fast (strengthened with a call_count assertion) and deletes tests for a long-applied, superseded data migration. No production code touched, diff matches description, and two bot reviewers already gave a thumbs up.
- 👍 on the PR from chatgpt-codex-connector[bot], hex-security-app[bot].
Gate mechanics and policy version
| Gate | Result | |
|---|---|---|
| prerequisites | ✓ | all clear |
| deny-list | ✓ | no deny categories matched |
| size | ✓ | 0L, 0F substantive, 100L/2F incl. docs/generated/snapshots — within ceiling |
| tier | ✓ | T0 auto-approve: T0-deterministic (100L, 2F, single-area, chore) |
| stamphog 2.0.0b3 | .stamphog/policy.yml @ 576c226 · reviewed head c64cbfc |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Trunk's slowest-tests report shows the statuspage retry tests dominating CI runner time.
TestFetchPage.test_retryable_statuses_raisemockssession.getbut leaves tenacity's@retrybackoff un-mocked, so each of the 5 parametrized status codes does 7 real exponential-backoff sleeps (2+4+8+16+32+60+60 ≈ 3 minutes of wall clock) on every CI run. It's the single largest source of wasted runner time in the suite.The same report flags
test_migration_0014.py. ItsTestMigrationsbase replays migrations backward and forward insetUpfor each of the class's 10 expanded test IDs, and the migration under test is an elidable one-off backfill with dozens of migrations shipped on top of it. It's long applied everywhere and has nothing left to guard.Changes
tenacity.nap.time.sleepviamonkeypatchintest_retryable_statuses_raise, so retries run instantly. The production retry config is untouched and the test still exercises the full retry count (now asserted viasession.get.call_count == 8) and the reraise behavior. All 5 cases drop from ~3 minutes each to milliseconds.products/warehouse_sources/backend/tests/test_migration_0014.py.How did you test this code?
Ran
pytest products/.../statuspage/tests/test_statuspage.py::TestFetchPagelocally: 7 passed in ~9s (previously the retry cases alone took ~15 minutes). The newcall_countassertion catches a regression no prior test did: someone reducing_MAX_ATTEMPTSor breaking the retry predicate would previously still pass, since the test only checked that the exception surfaced eventually. No coverage replaces the deleted migration test, deliberately, since the backfill has shipped everywhere.Automatic notifications
Docs update
No docs affected.
🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Claude (PostHog Code) authored this from a slowest-tests report review. Sonnet subagents investigated each slow test; these two were the cheap, high-confidence fixes (the funnel time-to-convert compare test and the stateright model tests need separate, more involved changes). Skills invoked: /writing-tests. Considered making the retry backoff configurable in production code instead, rejected as test-only plumbing in a production signature; the monkeypatch is self-contained and auto-restored.
Created with PostHog Code