Skip to content

test: disable TS background worker during migrations #287

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

joseph-sentry
Copy link
Contributor

we're getting the deadlock error in the API tests, the reason is that Timescale index creations deadlock with continuous aggregate refreshes like so:

2025-07-04 18:28:32.940 UTC [60026] DETAIL:  Process 60026 waits for AccessShareLock on relation 30135 of database 30126; blocked by process 60008.
        Process 60008 waits for ShareLock on relation 30150 of database 30126; blocked by process 60026.
        Process 60026: CALL _timescaledb_internal.policy_refresh_continuous_aggregate()
        Process 60008: 
                        CREATE INDEX IF NOT EXISTS ta_ts_testrun_summary_repo_id_ts_bin_idx
                        ON ta_timeseries_testrun_summary_1day (repo_id, timestamp_bin DESC);

so we have to disable the background worker while we run the migrations for tests

this PR takes overwrites the django_db_setup fixture and adds some statements to disable the background worker while we run migrations, similar to what we do in the sqlalchemy_db fixture in worker

@joseph-sentry joseph-sentry marked this pull request as ready for review July 4, 2025 19:57
@codecov-notifications
Copy link

codecov-notifications bot commented Jul 4, 2025

Codecov Report

Attention: Patch coverage is 85.71429% with 4 lines in your changes missing coverage. Please review.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
apps/codecov-api/conftest.py 85.71% 4 Missing ⚠️

📢 Thoughts on this report? Let us know!

Copy link

codecov bot commented Jul 4, 2025

Codecov Report

Attention: Patch coverage is 85.71429% with 4 lines in your changes missing coverage. Please review.

Project coverage is 94.33%. Comparing base (8030fd0) to head (1c8c073).

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
apps/codecov-api/conftest.py 85.71% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #287      +/-   ##
==========================================
- Coverage   94.33%   94.33%   -0.01%     
==========================================
  Files        1222     1222              
  Lines       45309    45337      +28     
  Branches     1441     1441              
==========================================
+ Hits        42744    42768      +24     
- Misses       2264     2268       +4     
  Partials      301      301              
Flag Coverage Δ
apiunit 96.47% <85.71%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@ElioDiNino ElioDiNino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will be hard to confirm things are working for certain since I found this to be a flakey issue, but generally looks good. I would suggest deploying this to staging and rerunning the workflow a few times to make sure the deadlock isn't popping up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants