docs backend evidence cd CD 03

CD-03 — Schema compatibility window and manifest floor

Status: LOCAL_DONE / INTEGRATION_PENDING Validation: local PASS; executor blockers resolved by owner decision (R1–R9); executor and live staging acceptance OWED Scope: Phase 3 prerequisite (the plan's "code change required first"). No rollout, backup, migration execution, landing switch, VPS, k8s, nginx or production action is included.

Delivered

• backend/db/migrations/schema_min_supported declares the oldest schema this build can run against (000132) and is embedded with the migration chain.
• backend/internal/platform/migrate now separates the compatibility rule from the database query. Check verifies that:
  • every known migration present in the database keeps its checksum;
  • no known migration up to the recorded maximum is missing;
  • the database is not older than schema_min_supported;
  • a newer database version is accepted as the additive side of the expand/contract window. Missing, modified or older-than-floor schemas still fail. Empty and malformed ledgers fail.
• The release workflow publishes schema_min_supported in abastevo-release-manifest-v1 and validates both its format and schema_min_supported <= migrations_max.
• The Phase 2 dry-run agent validates the field when present, carries it into the plan receipt target/current payloads, and remains compatible with older manifests that predate the field. New manifests always carry it.
• Unit tests cover the compatibility matrix; the PostGIS integration test now proves a modified ledger fails and a newer additive version is accepted.

Validation

• go test ./internal/platform/migrate/...: PASS (unit compatibility matrix and migration-name parsing).
• go test -tags=integration ./internal/platform/migrate -count=1 against the disposable PostGIS database: PASS, including the updated readiness test.
• make test-deploy-agent: 36 passed, 0 failed, including the new schema-floor format and above-chain refusals and the receipt floor assertion.
• make quick-verify: PASS (full selection, 86s).
• actionlint v1.7.12, scripts/check-release-workflow.sh, manifest fixture generation/validation and git diff --check: PASS.
• Secret scan: PASS; no host, IP, secret or operator path was added.

Expand/contract rule recorded for the next microphase

A new release whose schema_min_supported is above the previous release's floor contains a contract change. The apply-mode agent must refuse to run that migration automatically while the previous image is still the rollback target; additive releases keep the floor and may roll back. The field is now published so the gate can be implemented without another schema change.

Blockers resolved by owner decision (2026-10-27)

The Phase 3 blockers were caused by missing access, not by a real choice: this workspace could not see the staging host. A read-only inspection of the staging host on 2026-10-27 established the facts below; no state was changed.

Facts:

• Staging runs on single-node k3s (v1.35.4+k3s1) in one namespace. The API Deployment carries three containers (api, worker, private-storage), replicas: 1, RollingUpdate with maxSurge: 0 / maxUnavailable: 1, and images imported by hand with imagePullPolicy: Never. The web Deployment is Caddy serving the landing and portal from host directories. The database is a StatefulSet. A host nginx terminates TLS in front of the NodePorts.
• Namespace quota: limits.cpu 3500m of 4 used (API pod 2 CPU, DB 1, web 0.5), limits.memory 3840Mi of 8Gi.
• ghcr.io/alexandrezanata/abastevo-{api,worker,migrate,release} all allow anonymous pulls, and the staging host reaches ghcr.io.
• infra/compose.*.yml are the local, validation and future-production layouts; they are not the staging runtime.

Decisions:

ID	Decision
R1	The staging executor targets the real k3s namespace. Compose stays for local/validation and for a later production decision (G09); the executor keeps a runtime adapter boundary and implements only the k8s adapter now. Production stays manual.
R2	The live staging objects (API/web Deployments, Services, DB StatefulSet, quota, CronJob, Caddy ConfigMap) are captured into infra/k8s/staging/ as the reviewed source of truth, with placeholders for host paths and names and secretRef/configMapRef only (no secret values, IPs or operator paths). kubectl diff against the live namespace must be empty before the executor is enabled.
R3	Images are pulled from GHCR by digest (…@sha256:… from the signed manifest), imagePullPolicy: IfNotPresent, anonymously. Manual ctr imports end once the executor is enabled. The agent verifies the cosign identity and the provenance attestation before applying a digest.
R4	API and worker share one Deployment, so one rollout updates both containers (kubectl set image deployment/<api> api=<digest> worker=<digest>). Config gains ABASTEVO_API_CONTAINER and ABASTEVO_WORKER_CONTAINER; ABASTEVO_WORKER_DEPLOYMENT may equal the API Deployment.
R5	The owner raised the namespace limits.cpu from 4 to 6 on 2026-10-27 (previous quota backed up on the host; requests and memory unchanged). Headroom is now 2.5 CPU, enough for one extra 2-CPU API pod, so the target strategy is maxSurge: 1 / maxUnavailable: 0 (no gap). During a surge the old and new pods run two worker containers for up to the rollout timeout, so the executor microphase must first list every worker loop with its lock (advisory lock, lease or SKIP LOCKED) and prove two overlapping workers on a disposable PostGIS produce no duplicate effect. Until that proof exists, and whenever the live quota lacks room, the executor keeps maxSurge: 0 / maxUnavailable: 1 and records the measured gap.
R6	Order per release: verify manifest → schema-floor gate (a floor above the current one is a contract change and is refused automatically) → pg_dump backup verified with pg_restore --list and SHA-256 → migrate Job from the migrate digest (backoffLimit: 0, ttlSecondsAfterFinished) → set images → rollout status --timeout → smoke (/health/ready, one read per public route) → landing switch → write current.json. Any failure before current.json runs rollout undo, keeps the previous landing and writes a failure receipt. A migration is never rolled back automatically; the additive window keeps the old image valid.
R7	Landing ships inside the signed OCI release artifact as a landing-dist.tar.gz layer whose SHA-256 is in the manifest. The agent extracts it to <landing root>/releases/<sha>/ and switches a relative current symlink atomically; the web Deployment mounts the landing root and Caddy serves …/current, so a switch needs no pod restart. The previous release is kept for rollback; older ones are pruned to the last 5.
R8	Least privilege: the agent uses a kubeconfig for a ServiceAccount whose Role is limited to the staging namespace (get/patch Deployments, create/get/delete Jobs, get pods/logs, get quota, rollout undo). No cluster-admin, no other namespace. Host writes are limited by systemd ReadWritePaths to the state, receipt, backup and landing roots.
R9	Hygiene: every Job the executor creates sets ttlSecondsAfterFinished: 86400. Old finished pods left by manual jobs are cleaned once, only inside the staging namespace, after listing them in the receipt.

Acceptance (unchanged, now executable): two consecutive merges into main deploy themselves; a deliberately broken image is undone automatically and the previous release keeps serving; a contract-floor release is refused; a landing switch and its rollback both work. These runs are OWED to the next microphase.