Skip to content

2.31.0.0-b199

@pao214 pao214 tagged this 25 Jun 15:09
Summary:
#### Problem

Below is the log summary of a parallel query with vmodule=pg_session=2,pg_client_session=2

```
# Leader (PID 589293) launches 2 parallel workers
04:36:25.391 [589293] DEBUG: YB: launching parallel workers
04:36:25.392 [589293] DEBUG: YB: launched 2 parallel workers
# pg_session.cc: Leader sets read_time from follower read staleness
04:36:25.391 589293 pg_session.cc:900 Perform options:
    read_ht: 7281091302930841600 (physical: 1777610181379600)
    read_from_followers: true, read_time_serial_no: 50
# pg_session.cc: Worker 1 independently computes its own read_time (+46ms)
04:36:25.459 589717 pg_session.cc:900 Perform options:
    read_ht: 7281091303120982016 (physical: 1777610181426021)
    read_from_followers: true, read_time_serial_no: 50
# pg_session.cc: Worker 2 independently computes its own read_time (+56ms)
04:36:25.460 589718 pg_session.cc:900 Perform options:
    read_ht: 7281091303160123392 (physical: 1777610181435577)
    read_from_followers: true, read_time_serial_no: 50
# pg_client_session.cc: tserver receives all three distinct read_times
# through the SAME session (pid 589293), confirming it uses whatever
# read_time each worker specified:
04:36:25.393 Session id 6 (pid 589293): read_time={ physical: 1777610181379600 }  # leader
04:36:25.460 Session id 6 (pid 589293): read_time={ physical: 1777610181426021 }  # worker 1
04:36:25.460 Session id 6 (pid 589293): read_time={ physical: 1777610181435577 }  # worker 2
# All workers finish
04:36:25.470 [589293] DEBUG: YB: all 2 parallel workers finished
```

From the log, we can see that all the workers use different read time. This can potentially cause an inconsistent read.

#### Root Cause

Follower reads computed their read time (now - staleness) in the YSQL backend
using the backend's own clock. Each parallel worker is a separate backend, so
workers in the same query computed different read times and could return
inconsistent results.

#### Fix

Pick the read time in PgClientSession. When the PgClientSession picks the read point it lowers it by that
staleness (SetFollowerReadTime). Every perform and
parallel worker that shares the serial number - reads at the same time.

Moreover, clock usage is removed from PgTxnManager. This is a step forward in removal of HybridClock
from postgres backend since postgres backend is not involved in hybrid time propagation.
PgApiImpl keeps a HybridClock only for ANALYZE sampling; see #16034.

**Upgrade/Rollback safety:**

Only changes proto used for communication between postgres and local tserver proxy.

Test Plan:
Jenkins

./yb_build.sh release --java-test 'org.yb.pgsql.TestPgFollowerReads#testBankInvariantWithParallelFollowerReads'

Fails with the error below without the fix

```
INCONSISTENCY: expected total=100000 but got 100004 (diff=+4)  after 611 transfers
INCONSISTENCY: expected total=100000 but got 99993  (diff=-7)  after 742 transfers
INCONSISTENCY: expected total=100000 but got 99988  (diff=-12) after 1065 transfers
INCONSISTENCY: expected total=100000 but got 99999  (diff=-1)  after 1213 transfers
INCONSISTENCY: expected total=100000 but got 100007 (diff=+7)  after 1372 transfers
INCONSISTENCY: expected total=100000 but got 100007 (diff=+7)  after 1645 transfers
```

Reviewers: pjain, amartsinchyk, sanketh, smishra, dmitry

Reviewed By: pjain, dmitry

Subscribers: mtakahara, ybase, yql

Differential Revision: https://phorge.dev.yugabyte.com/D52581
Assets 2
Loading