Optimize BGW CI tests #7223
Optimize BGW CI tests #7223
Conversation
This change helps optimize two bgw tests, by a) reducing the time a test spends waiting for scheduled jobs and b) decreasing the amount of data inserted into the table. This should have no effect on the efficacy of the tests themselves.
|
These are the improvements I saw when running this locally:
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #7223 +/- ##
==========================================
+ Coverage 80.06% 81.78% +1.71%
==========================================
Files 190 205 +15
Lines 37181 38320 +1139
Branches 9450 9936 +486
==========================================
+ Hits 29770 31341 +1571
+ Misses 2997 2987 -10
+ Partials 4414 3992 -422 ☔ View full report in Codecov by Sentry. |
| COMMIT; | ||
| END | ||
| $$; | ||
| CREATE OR REPLACE PROCEDURE custom_proc2(jobid int, config jsonb) LANGUAGE PLPGSQL AS | ||
| $$ | ||
| BEGIN | ||
| UPDATE custom_log set msg = 'msg2' where msg = 'msg0'; | ||
| perform pg_sleep(10); |
There was a problem hiding this comment.
It doesn't look like custom_proc2 waits for anything in this test so I eliminated this sleep call entirely.
There was a problem hiding this comment.
The problem here is actually such kind of test should be an isolation test instead. This sleep is to make sure we'll end up with a concurrent update issue: https://github.com/timescale/timescaledb/blob/main/tsl/test/expected/bgw_job_stat_history_errors.out#L96
There was a problem hiding this comment.
We still see the same error -- I don't see why custom_proc2 needs to sleep for that to happen. Doesn't the concurrent update issue occur immediately when custom_proc2 tries to update a row that custom_proc1 already updated?
I agree that ideally this should be an isolation test.
| @@ -549,7 +549,7 @@ INSERT INTO sensor_data | |||
| random() AS cpu, | |||
| random()* 100 AS temperature | |||
| FROM | |||
| generate_series(:'start_date_sd'::timestamptz - INTERVAL '1 months', :'start_date_sd'::timestamptz - INTERVAL '1 week', INTERVAL '1 minute') AS g1(time), | |||
There was a problem hiding this comment.
These inserts were taking a significant amount of time, so I reduced the amount of data being generated. I don't think this affects test behavior itself.
There was a problem hiding this comment.
I'm OK with doing this as long as it doesn't affect the flakiness of the test i.e. makes it more prone to failure.
Can you run the CI here multiple times to ensure that isn't happening? I say this because bgw_custom is notorious for being flaky and we don't want to make the situation worse here.
This change helps optimize two bgw tests, by a) reducing the time a test spends waiting for scheduled jobs and b) decreasing the amount of data inserted into the table. This should have no effect on the efficacy of the tests themselves.
Disable-check: force-changelog-file