Skip to content

2025.2.5.0-b9

2025.2.5.0-b9: [BACKPORT 2025.2][#31895] YSQL: Fix dangling ConditionVariable pointer
@andrei-mart andrei-mart tagged this 23 Jun 14:19 
Summary:
Original commit: c9cc15318224068a0c4a9afe2756d03cd0a34e4a / D53954
There's rare parallel query error handling scenario involving multiple parallel workers where a dangling ConditionVariable pointer may be left in the system.
The sequence of events is as follows:
1. A parallel scan exhausts the parallel ranges in the shared memory buffer;
2. A background worker comes for a parallel range, the buffer is empty, so worker starts fetching a new set of parallel ranges;
3. The main worker comes for a parallel range, the buffer is empty and fetch is in progress, so worker sleeps on associated ConditionVariable. As a part of the sleep the pointer to the ConditionVariable is saved;
4. Other background worker encounters an error and send it to the main worker;
5. Main worker's sleep on ConditionalVariable is interrupted by the data from the background worker. The data is an error message, so the main worker reraises it and execution longjumps to the backend's event handling loop;
6. The main worker attempts transparent retry. There's active PQ, so the main worker calls YbClearParallelContexts() to prevent further interruptions from the parallel workers. As a part of the cleanup the PQ's dynamic shared memory block is released and that makes the pointer saved on step 3 dangling.

The solution is to cancel sleep in YbClearParallelContexts while dynamic shared memory block is still here.

D53426 increased probability if the issue. Previously the parallel query was cleaned up only after "retryable" errors, now they are cleaned up after any error.

Test Plan:
Stress tests that have previously shown the failure often now all good:
https://stress.dev.yugabyte.com/auto_runs/3a057729-968b-4ab0-9a0d-db1a654b1b19
https://stress.dev.yugabyte.com/auto_runs/b66c7bcc-2a96-4ab2-ab48-a4527803f1b4
https://stress.dev.yugabyte.com/auto_runs/88b239ad-e0f1-4183-8c94-351dde233ad7

Reviewers: patnaik.balivada, pjain, #db-approvers

Reviewed By: patnaik.balivada

Subscribers: yql

Tags: #jenkins-trigger

Differential Revision: https://phorge.dev.yugabyte.com/D54567
Assets 2
Loading