Slice 3A: processgit-updater sidecar (state machine + HTTP API + cosign verify)

[rg4444]

Slice 3A — processgit-updater sidecar

Foundation of the in-product self-update story. Adds the updater/ sidecar: a tiny separate Go module (stdlib-only) that orchestrates ProcessGit updates inside Docker deployments.

Why a separate process

1. A container can't safely update itself in place. Replacing the running binary while it serves requests races with active sessions. The sidecar lives outside the main app's lifecycle.
2. Privilege boundary. The updater needs /var/run/docker.sock. The main ProcessGit container does not.
3. Tiny dependency surface. Stdlib-only Go (enforced by TestNoExternalImports), plus docker-cli + cosign in the runtime image. ~150 MB final image, dominated by the docker CLI.

What ships

Layer	What
HTTP API	GET /healthz, GET /status, GET /releases/latest, POST /update, GET /update/{id}, GET /history. All non-/healthz paths require Authorization: Bearer $PROCESSGIT_UPDATER_TOKEN, compared in constant time.
State machine	idle → planning → snapshotting → pulling → verifying → migrating → swapping → healthchecking → committed. Failure paths: rolling_back → rolled_back (recovered) or failed (manual intervention). One job at a time, enforced by Store.AddJob.
Persistence	$STATE_DIR/state.json, atomic write-temp-then-rename, bounded history (50 jobs).
GitHub client	Real api.github.com/repos/…/releases + asset downloads.
Cosign	Real cosign verify (image) + cosign verify-blob (release.json), via os/exec.
Runtime image	Multi-stage Dockerfile: golang:1.25-alpine3.22 build → alpine:3.22 runtime with docker-cli, cosign (from gcr.io/projectsigstore/cosign:v2.4.1), ca-certificates, tini. Non-root, EXPOSE 9000.

Critical safety property

The manifest signature is verified before we trust any of its fields (image ref, digest, migration command). An attacker who can substitute a malicious release.json cannot redirect the updater to a different image, because cosign verify-blob against the workflow's OIDC identity must pass first.

What's stubbed (Slice 3B)

Docker.Pull, Docker.SwapContainer, Docker.RunMigration, Docker.Healthcheck, Docker.Rollback, Docker.InspectAppImageDigest, Docker.Snapshot — all currently log and sleep. Stub mode is the default (PROCESSGIT_UPDATER_STUB=true). The orchestrator state machine, GitHub client, cosign verification, persistence, and HTTP API all run real code; only the container surgery is deferred.

This split keeps the PR reviewable (~1400 lines of Go, no real container operations) and lets the state machine and signature-verification logic harden before they touch live containers.

Tests

7 tests, all passing locally against Go 1.22 (sandbox limit) and against Go 1.25 (target):

=== RUN   TestStore_RoundTrip                  --- PASS (0.00s)
=== RUN   TestState_IsTerminal                 --- PASS (0.00s)
=== RUN   TestJob_TransitionFinishesOnTerminal --- PASS (0.00s)
=== RUN   TestOrchestrator_HappyPath           --- PASS (14.03s)
=== RUN   TestOrchestrator_RejectsConcurrent   --- PASS (0.00s)
=== RUN   TestAPI_BearerAuth                   --- PASS (0.00s)
=== RUN   TestNoExternalImports                --- PASS (0.00s)
PASS
ok    github.com/Algomation-AI/ProcessGit/updater   14.043s

The 14-second HappyPath test traverses every state in the machine using stubbed docker ops, a fake GitHub server (httptest.NewServer), and /bin/true as a cosign substitute.

Local quickstart

cd updater
PROCESSGIT_UPDATER_TOKEN=devtoken \
PROCESSGIT_UPDATER_STATE_DIR=/tmp/pg-updater \
go run .

# In another terminal:
TOKEN=devtoken
curl -s http://localhost:9000/healthz | jq
curl -s -H "Authorization: Bearer $TOKEN" \
     -H "Content-Type: application/json" \
     -d '{"target_tag":"v0.1.0"}' \
     -X POST http://localhost:9000/update | jq

Full state-machine run completes in ~20 seconds in stub mode.

Out of scope (queued)

When	What
Slice 3 follow-up	deploy/docker-compose.yml integration — adds the updater service, wires bearer token via .env, sets internal network
Slice 3 follow-up	.github/workflows/release.yml extension — builds & signs processgit-updater image paired with the main image on every semver tag
Slice 3B	Real docker pull / swap / healthcheck / rollback
Slice 3C	Volume snapshot + restore for disaster recovery
Slice 3D	Robust migration runner
Slice 4	Admin UI at /-/admin/updates consuming this API

File inventory

updater/
├── Dockerfile          (89 lines)  multi-stage build + runtime
├── README.md           (166 lines) architecture, config, ops, scope
├── api.go              (159 lines) routes, auth middleware, handlers
├── cosign.go           (112 lines) cosign verify wrapper
├── docker.go           (122 lines) docker CLI wrapper (stubbed)
├── go.mod              (8 lines)   separate module, stdlib only
├── job.go              (268 lines) Job/Step types, atomic store
├── main.go             (169 lines) entrypoint, env config, signal handling
├── manifest.go         (266 lines) release.json types + GitHub fetcher
├── orchestrator.go     (200 lines) state machine
└── orchestrator_test.go (346 lines) tests
                       ─────
Total                   1,905 lines

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ff62b6507b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Pin cosign identity policy outside untrusted manifest

VerifyBlob uses m.Signing.IdentityRegex and m.Signing.Issuer from release.json before that manifest is trusted, which defeats the stated security boundary. If an attacker can substitute release assets, they can choose a permissive regex/issuer and provide a matching cert/signature so blob verification passes, then control image and migration fields. The expected signer policy must be fixed in updater config/code (or another trusted channel), not read from the unverified payload being authenticated.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Authenticate release asset downloads with GitHub token

The updater documents PROCESSGIT_UPDATER_GITHUB_TOKEN as required for private repos, but downloadAsset does not send any Authorization header when fetching browser_download_url. In private repositories this causes manifest asset fetches to fail (typically 404), so updates cannot start even when a token is configured. Reuse the token-authenticated API asset flow (or equivalent authenticated download path) for these requests.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Return a snapshot from Active to avoid concurrent races

Store.Active returns the internal *Job pointer directly, unlike Get/List which copy. While an update is running, /status can serialize this object concurrently with orchestrator mutations (transitionTo appends/modifies Steps), creating unsynchronized read/write access to the same struct. This can yield race-detector failures and undefined runtime behavior; return a copied snapshot under the lock instead.

Useful? React with 👍 / 👎.