Define adjacent-release HA update qualification - #892
Draft
ankitgoswami wants to merge 52 commits into
Draft
Conversation
🔐 Codex Security Review
Review SummaryOverall Risk: NONE FindingsNo findings. NotesThe authoritative diff is documentation-only. The qualification procedure remains fail-closed: every gate must pass before release promotion. No runtime, protobuf, database, pool configuration, or credential-handling code changed. Generated by Codex Security Review | |
This was referenced Aug 7, 2026
ankitgoswami
force-pushed
the
ankitg/ha-adjacent-update-qualification
branch
from
August 7, 2026 18:44
0004870 to
d8d4b8e
Compare
ankitgoswami
force-pushed
the
ankitg/ha-adjacent-update-qualification
branch
2 times, most recently
from
August 7, 2026 18:59
c0e905e to
67ad6fa
Compare
ankitgoswami
force-pushed
the
ankitg/ha-adjacent-update-qualification
branch
from
August 7, 2026 19:11
67ad6fa to
fa15389
Compare
ankitgoswami
force-pushed
the
ankitg/ha-adjacent-update-qualification
branch
2 times, most recently
from
August 7, 2026 19:27
0cf4fb7 to
3ea1c82
Compare
ankitgoswami
force-pushed
the
ankitg/ha-adjacent-update-qualification
branch
2 times, most recently
from
August 7, 2026 19:45
52b0191 to
f60cfb3
Compare
ankitgoswami
force-pushed
the
ankitg/ha-adjacent-update-qualification
branch
from
August 7, 2026 19:47
f60cfb3 to
8a99b7e
Compare
ankitgoswami
force-pushed
the
ankitg/ha-adjacent-update-qualification
branch
from
August 7, 2026 19:52
8a99b7e to
1367cee
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reviewable diff: +222/-0 across 2 files (excludes generated, test, and story files).
Summary
Defines the evidence required to support an adjacent-version HA application update from stable
Nto prereleaseN+1. The same target tag and assets become stable only after migration compatibility, passive-first update, mixed-version forced takeover, bounded completion and timeout recovery, bidirectional failover, command recovery, infrastructure preservation, deployment-swap, forward-activation, updater interruption, and reboot gates pass on both amd64 and arm64.Stack: #887 -> #888 -> #889 -> #890 -> #891 -> #892. This documentation-only diff is relative to completion PR #891.
How it works
Operators publish the final
N+1tag and assets at the fixed official origin as a GitHub prerelease. HA does not offer UI-triggered updates, so qualification configures the root-owned updater environment to authorize only that exact target and then uses the explicit local operator command. The authorization is restored after every clean reinstall, verified before each update attempt, and removed after the run. The report requires clean-install qualification for the exact source and target artifacts and verifies that the target's reviewed qualification file and manifest metadata name the exact source. It first runs compatibility tests, then loads a sanitized production-scaleNdataset into the HA cluster so the real passive updater runs migrations under sustained traffic. Gap-free active-health and VIP ownership recording starts before the passive update. Authenticated reads and uniquely identified commands continue through passive-first update and completion, with bounded success gaps; every background command must reach SUCCESS, except the deliberately interrupted PROCESSING rows. The live handoff carries oneNPROCESSING row and PENDING successor intoN+1, proving the interrupted row fails without replay and the successor dispatches. A separate exact-artifact run pauses after the old application stops, makes the updated peer unavailable, proves the updater enters its takeover wait, and requires automaticNrecovery within 60 seconds. Rename-window and self-update faults require direct process and filesystem evidence before injection rather than inferring the window from an ambiguous durable marker. A client loaded fromNmust still work against theN+1VIP. Another cleanNbaseline forces the updatedN+1passive to take over before completion, then updates the oldNhost through the ordinary passive command. Public and database-backed probes measure normal interruption below 15 seconds; container identity, PostgreSQL start times, updater checks, and one-at-a-time reboots finish the evidence. Forward-activation interruption must resume target startup and migrations; self-update interruption passes only when the updater converges to the same version as the retained application deployment. Only after every gate passes does a final drift guard re-resolve both commits, re-download the complete source and target asset sets, compare their recorded digests, and promote the unchanged prerelease to stable.flowchart LR A["Verify N and N+1 artifacts"] --> P["Update passive"] P --> M["Exercise mixed versions"] M --> X["Force mixed-version takeover"] X --> C["Complete active within 15s"] C --> F["Fail over both directions"] F --> R["Reboot and verify"] R --> V{"Every gate passes?"}Areas of the code involved
deployment-files/ha/UPDATE_QUALIFICATION.mddeployment-files/ha/README.mdKey technical decisions & trade-offs
Testing & validation