Repository navigation
1. Reconnects no longer starve HTTP responses under a small CPU quota
Previously, pg-status could open connections to several hosts at once: at startup, after a network failure, while hosts were unavailable, and when connections were recycled. TLS and SCRAM authentication make connection setup far more expensive than a status query.
Now new connections take turns:
- Only one connection negotiates TLS or authenticates at a time. Waiting for a TCP connection does not count, and one setup holds the turn for at most 1 second, so unreachable or hanging servers do not hold up the others.
- After a setup that reached a TLS handshake or password authentication, successful or not, the next one waits 100 ms, leaving quota for the HTTP thread.
- Connection starts are spaced by
min(pg_status__sleep_ms / number of hosts, 250 ms). - An aged connection is replaced only when the turn is free and no host with a missing or broken connection is waiting. Until then, it keeps serving checks. A check is never delayed because of the age of a working connection.
While a master is selected, dead hosts whose connection attempts fail after a TLS handshake or password authentication, for example at a connection pooler whose PostgreSQL is down, are checked less often. The period doubles with each such failure, up to 8 seconds (2, 4, then 8 s with the defaults), or pg_status__sleep_ms if that is longer. Hosts that fail earlier, such as refused or timed-out TCP connections and DNS errors, are still checked every pg_status__sleep_ms. While no master is selected, every dead host is checked every pg_status__sleep_ms, because any of them may bring the master back. Already postponed checks return to the normal period as soon as the master is lost.
When upgrading, check the following:
- The default
pg_status__conn_max_age_mshas changed from300000(5 minutes) to3600000(1 hour) to reduce costly reconnects. pg-status resolves host names only when it connects. If a host entry points to a proxy, pooler, or DNS alias that can move to another server while keeping existing sessions, the reported role can stay stale for up to this long. In that case, set a lower value explicitly. - With many hosts,
/readyand the host-information endpoints become available later, because the initial connections take turns. Each host adds the larger ofmin(pg_status__sleep_ms / number of hosts, 250 ms)and one session setup plus 100 ms. With dozens of hosts, this is several seconds; size readiness probe timeouts accordingly. - When several hosts fail at once, their reconnects take turns, so outage detection can take longer. The effect is largest when servers accept TCP connections but never complete session setup: with four such hosts and the defaults, detection takes about 9 seconds instead of 4.
- While a master is selected, pg-status may notice up to 8 seconds later that a host failing after the TLS handshake has recovered.
pg_status__query_timeout_msno longer includes the time a ready TCP connection waits for its session setup turn.
2. Time lag of caught-up replicas no longer grows while the master is idle
Previously, lag_ms was only the age of the last transaction a replica had replayed. While the master committed nothing, this age kept growing even though the replica had nothing left to replay. After pg_status__sync_max_lag_ms (1 s by default) without commits, every replica became time-unsynchronous, and /sync_by_time, /sync_by_time_and_bytes and /replica?lag_ms=... sent all reads to the master. On a quiet database, the master served all reads.
Now, when a replica has replayed exactly the WAL position from the selected master's last successful check, lag_ms is also bounded by the time since that check started. This is less than pg_status__sleep_ms while checks run on schedule. To use the master's latest position, a replica's check may wait for a master check that is in progress or due; the wait neither fails the check nor delays the next one. The bound does not apply, and lag_ms stays the commit age, in these cases:
-
no master is selected, or its last check failed;
-
the replica's position or timeline differs from the master's;
-
the master has WAL it has not written yet: asynchronous commits, an open transaction that has written data, or hint bits with data checksums.
-
Read-your-writes is now safe across failovers. Previously, a replica left on the WAL history that the new master abandoned could be past
min_lsnwithout having the write. Now a replica whose timeline differs from the master's meets neithermin_lsnnor byte thresholds. The newtimelinequery parameter, accepted by/replica,/sync_by_*and/most_sync_by_bytes, matches only replicas known to be on the timeline of the write. It also protects reads before pg-status selects the new master. The recommended recipe has changed: after COMMIT returns, readpg_current_wal_insert_lsn(), becausepg_current_wal_lsn()may not cover an asynchronous commit, and take the timeline from the WAL file name. See the README. -
Byte lag now uses the WAL position of the selected master, or of the last selected one while none is selected. Previously, any host that reported itself as a master could set it, such as the second master in a split brain.
-
A lag that cannot be measured is now
nullin/hostsand/status, with afalsesync flag, and meets no threshold. Previously, a replica that had not replayed a transaction since it started reportedlag_ms: 0and looked synchronous, however far behind it was.lag_bytesisnullwhen there is no master position to compare with or when the replica's timeline differs from the master's.
When upgrading, check the following:
- The polling query now also calls
pg_control_checkpoint()andpg_control_recovery(), which PostgreSQL allows everyone to use by default. On a permission error, pg-status logs a warning and polls without WAL receiver statistics. If that fails too, it also polls without the control file functions. In that case replica timelines are unknown, and only thetimelineparameter protects read-your-writes after a failover. - Grant
pg_monitororpg_read_all_statsto the monitoring user. Without them, a replica that already follows the new master after a failover meets no byte threshold and nomin_lsnfor up tocheckpoint_timeout, and such reads go to the master. - On a master without writes, a caught-up replica's
lag_mscan be anywhere belowpg_status__sleep_ms. A smaller threshold, such aslag_ms=50, excludes such a replica part of the time. Usemin_lsnfor an exact guarantee. - Clients that read
lag_msandlag_bytesfrom/hostsor/statusmust handlenullfor live replicas. - Because a replica's check may wait for the master's, its result can be published up to one polling period later. The first
/readycan also take up to one period longer.