AG suspension semantics: one rule that holds whether or not the docs are right (#991)

[erikdarlingdata]

Doc-only, no behavior change — but it corrects an under-description of the SUSPENDED state that affects how every AG panel and threshold should be read, including one measure #1695 shipped an hour ago.

@ag-alerts-builder suggested capturing how the four new *_time columns behave under suspension "while the fixture is still up, because the pattern here was that the vendor docs were wrong about exactly this state." They were right to ask.

What I measured

On the live Docker AG fixture (SQL Server 2022, CLUSTER_TYPE = NONE), write load applied, ALTER DATABASE ... SET HADR SUSPEND on the secondary, sampled from the primary's view of the remote row every 15s:

t	susp	sync	lag_s	send_q	redo_q	commit_t	hardened_t	redone_t	received_t	est_redo_min	est_send_min
0s	1	NOT SYNCHRONIZING	30	NULL	21896	19:08:26.687	19:08:26.930	19:08:26.687	NULL	0.014356	NULL
15s	1	NOT SYNCHRONIZING	45	NULL	21896	19:08:26.687	19:08:26.930	19:08:26.687	NULL	0.014356	NULL
30s	1	NOT SYNCHRONIZING	60	NULL	21896	19:08:26.687	19:08:26.930	19:08:26.687	NULL	0.014356	NULL
45s	1	NOT SYNCHRONIZING	75	NULL	21896	19:08:26.687	19:08:26.930	19:08:26.687	NULL	0.014356	NULL

Everything freezes except secondary_lag_seconds, which accrues — an independent second confirmation of @ag-fixture-builder's finding, on a separate run.

Why it matters more than "one more quirk"

The columns disagree in opposite directions. secondary_lag_seconds grows while suspended (over-reporting); the four *_time columns freeze, so a cross-replica commit-time delta stops growing exactly when replication has stopped — understating the problem at the moment it is worst. Averaging them, or cross-checking one against the other without reading is_suspended first, produces nonsense.

The sharpest edge is a measure this release just shipped. est_redo_completion_time_min is queue ÷ rate, and with both operands frozen it holds a small, static, reassuring value — 0.0144 min, flat across the whole suspension — when the honest answer is "never, movement is stopped." A suspended replica makes a drain panel look healthier than reality. est_send_drain_time_min at least reads NULL, because its queue goes NULL, so the NULLIF guard saves that one by accident rather than by design.

The fix: a rule instead of a new absolute

I could have flipped "reads 0" to "accrues" and moved on. @ag-alerts-builder proposed a better shape, which I've adopted verbatim in spirit:

A suspended row may RAISE an alarm but may never CLEAR one.

That is correct under both the documented and the measured behavior, so nothing downstream has to bet on which is true — which matters, because my evidence is one build on a clusterless AG with WSFC untested. Stating an absolute I've verified twice on one configuration is how the original MS Learn sentence became wrong in the first place. The rule now leads the collector's doc block, and the compose-measure comment mirrors it with the drain-estimate specifics.

Two more things worth writing down

• last_received_time read NULL in every sample, healthy and suspended alike, on this fixture. Treat it as optional, not expected.
• last_commit_time / last_redone_time sit still on an IDLE database. They are "time of last commit", not a heartbeat. So now - last_commit_time is not a lag measure: on a quiet, perfectly healthy replica it grows without bound. I saw 7 minutes of apparent "lag" on a caught-up secondary reporting secondary_lag_seconds = 0. That is a trap someone will otherwise walk into the first time they use these columns.

Fixture hygiene

Borrowed, restored, verified: both replicas back to SYNCHRONIZED / is_suspended = 0 / lag 0, and the dbo.ag_probe load table I created was dropped (OBJECT_ID confirms NULL). Left exactly as found.

Tests

Darling.Tests   3251 passed, 0 failed (156 skipped - the live-Postgres gated set)
Solution build  0 warnings, 0 errors

No test changes: this touches XML doc comments and one block comment only.

🤖 Generated with Claude Code