Cut the deploy over to the artifact store: image fetch, deployed markers, prewarm retired

[anth-volk]

Fixes #652

Phase 3 of the baseline artifact pipeline (phase 1: #644, phase 2: #649). The store that phases 1–2 fill and verify finally gets its consumer: the image fetches precomputed artifacts instead of building datasets, and the deploy records what it shipped.

What changed

Commit 1 — self-contained fetch_artifacts (inert). New image-layer function in _image_setup.py: downloads every manifest-listed artifact into POLICYENGINE_DATA_FOLDER, with a freshness gate (manifest receipt versions vs the image's env versions, plus the installed bundle receipt's data version when present), one loud error listing all missing store objects before any download, unconditional overwrites, and GCS credentials built in memory from the secret blob (service_account.Credentials.from_service_account_info passed straight to storage.Client — nothing is ever written to disk, since a run_function layer commits its filesystem into the image; reworked in the follow-up commit after review). Zero callers in this commit.

Commit 2 — the atomic swap. app.py resolves the deploy manifest on the runner at modal.is_local() time from the new POLICYENGINE_MANIFEST_DIGEST env var (threaded from needs.precompute.outputs.manifest_digest, alongside the bucket var and GCP_CREDENTIALS_JSON) and passes bucket + manifest content as the fetch layer's args — the layer cache follows the artifact set exactly, retiring the force_build ritual. A digest-less import stays safe (the precompute and smoke apps import app.py where no digest can exist) via a None-manifest sentinel that fails at build time, never import time. The prebuild layer, PREBUILD_DATASET_YEARS, and prewarm_app.py are deleted; retired tests are replaced per a migration mapping (fetch layer ordering + sentinel + years-cover-default; the writer==reader filename coupling was already locked by the phase-1 precompute tests).

Commit 3 — deployed markers. A "Record deployment marker" step runs on both legs after the health check, writing deployed/<environment>.json (manifest digest, versions, run identity) via the first caller of the phase-1 marker helpers — the liveness signal for phase-4 GC. A failed marker write fails the deploy job by design.

Commit 4 — docs. Executor README describes the pipeline as it now is (fetch layer, freshness gate, local-deploy inputs, markers, the inherited bundle-install staleness residual); all remaining prebuild/prewarm prose is gone.

Why the merge is atomic

Once app.py expects the manifest digest, a deploy without the precompute output fails. The producer already flows from phase 2, so this PR adds only the consumer — there is no intermediate merge state where a deploy half-consumes the store.

Verification

• Full executor suite green after every commit (391 tests at head; the six retired prebuild tests are accounted for in the migration mapping in the plan).
• Workflow YAML parses; the deploy job's diff is exactly three env lines plus the marker step; residue grep for prebuild/prewarm is clean repo-wide.
• Not run at PR time (by design — nothing precomputes or deploys from PRs): the real proof is the post-merge deploy. Gate: both legs green with fetch-in-seconds image builds, baseline_artifact=hit on a beta economy request with p50 down, deployed/beta.json + deployed/prod.json in the bucket, then a fault drill (delete one baseline object → local fetch fails loudly → plain precompute re-run heals with an unchanged manifest digest).

Rollback

One revert restores the prebuild layer, prewarm app, workflow, and README verbatim; phases 1–2 keep running harmlessly underneath. Markers go stale after a revert (last-writer-wins, no more writers) — the phase-4 GC design must treat marker staleness conservatively.

🤖 Generated with Claude Code