Add artifact-pipeline keys, store, runtime reader, and precompute app#644
Merged
Conversation
The exponential backoff in the segmented-national collect loop capped at 15s, so a lone child completing after a quiet stretch could sit unnoticed for up to a full 15s before the parent's next probe. Cap at 8s to trim that worst-case collection lag; the walk is now 0.5 -> 1 -> 2 -> 4 -> 8. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Identity module for the baseline artifact pipeline: canonical-JSON sha256 keys for single-year dataset artifacts (ds1) and precomputed baseline simulations (bl1), the deterministic Simulation id derived from the baseline digest, and the store path layout. Keys digest the full input closure (versions, data content sha, certification fingerprint), so key equality means byte-identical artifacts and staleness cannot exist as a state. Golden pins turn any digest or upstream cache_key format change into a reviewable diff instead of silent cache churn. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Write-once uploads (if_generation_match=0, PreconditionFailed reported as already-present success — identical keys mean identical bytes, so the loser of a concurrent write lost nothing), last-writer-wins deployed markers, and content-addressed manifest read/write. google-cloud-storage becomes an explicit dependency of both the project and the image group instead of a transitive ride-along via policyengine. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The runtime read path of the artifact pipeline. Qualifying baselines
(current-law policy, macro scope, default data, national or region-group
scope) get a deterministic Simulation id derived from the artifact key, so
ensure() finds a precomputed {id}.h5 beside the dataset when one is baked
into the image. ArtifactBaselineSimulation trusts a load only after
diffing resolve_entity_variables against the loaded frames — any gap
(cliff or LSR columns, stale artifact) falls back to run()+save(), so an
artifact can cost compute but never correctness. With no artifacts
present, behavior is unchanged (miss -> run), which is what makes this
safe to ship ahead of the writer. The outcome (hit/incomplete/miss) is
logged and exported as the baseline_artifact attribute.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The write half of the artifact pipeline, run via modal run (no deploy): wave 1 builds the single-year US datasets (one container per year, 2026/ 2027/2025), wave 2 computes the 20 per-cohort national baselines per year on containers shaped like run_simulation_segment. All identity collection and store I/O happen in containers (the bundle receipt the keys digest lives in the image, not on the deploying machine); baselines are built through the executor's own _resolve_region + _build_simulation on a synthetic child request, so writer and reader identities agree by construction — and a test pins it. Content-addressed planning makes re-runs no-ops; --force recomputes; a determinism gate loads the uploaded artifact and compares it frame-by-frame against an independent fresh run. Prints MANIFEST_DIGEST=<digest> for the deploy pipeline to consume. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The precompute app had outgrown the self-contained-ephemeral-app shape of smoke_app/prewarm_app: it carried real library logic. Function bodies, planning, manifest assembly, and the local-entrypoint orchestration move to policyengine_simulation_executor.precompute, mirroring the deployed app's split (run_simulation -> run_simulation_impl); the app file keeps only image, declarations, and thin wrappers. The orchestration takes its five Modal function handles as parameters, so the whole flow — miss-only spawning, the determinism-gate probe, the abort-before-publish path, and the MANIFEST_DIGEST= stdout contract the deploy job will parse — now has direct test coverage with plain fakes. The fake-modal scaffolding the app tests still need is shared from fixtures/fake_modal.py instead of being copied per test file, and the dataset builder now goes through _country_module like the rest of the runtime. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two review gaps. The runtime must pass the resolved region code into _build_simulation — it feeds the deterministic-id predicate, and losing it would silently disable artifact reuse (every baseline a miss) with the suite still green; the impl-core test now asserts it. And the validate-on-load guard is now exercised against real h5 files through the real MicrosimulationModelVersion load/save implementations: a missing artifact raises FileNotFoundError end-to-end (the clean miss branch), a saved artifact loads as a hit with weight columns intact, and an on-disk artifact missing a requested column recomputes and overwrites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
README gains the artifact-precompute section (bucket env var, run command, idempotency, the write-once --force caveat, the manifest-digest stdout contract). The qualifying predicate's docstring now states that it is deliberately broader than the store's contents. Test docstrings and names stop using 'pin' for test coverage — a pin fixes a version value; goldens and regression tests are what these are. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Decision from review: store objects are never overwritten by anything, including --force. Healing a bad artifact is always delete-the-object, then re-run — deletion turns the key back into an ordinary miss. The README now states this as the deliberate contract (race safety, auditability, clobber protection) rather than a caveat. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
Phase 1 gate: passed (staging, bucket First run against the empty store:
Immediate re-run:
Both gate criteria from the plan are met (populated store + measurable no-op). |
Every structure crossing a precompute function boundary — the plan and its entries, the bundle receipt, the deploy manifest, and the worker results — is now a strict Pydantic model (extra=forbid, matching the gateway contract's discipline) in precompute_models. Plain dicts remain only at the Modal serialization boundary, where the app wrappers validate inbound and dump outbound, so planner/worker shape drift fails loudly at the edge as a ValidationError instead of a KeyError somewhere mid-computation. The typing collapsed the redundant signatures it exposed: workers now take their plan entry instead of (year, group, expected) triplets, and the id-mismatch check consolidates into _prepare_cohort_baseline. The manifest is content-addressed, so its canonical payload must not change shape: a wire-compatibility test asserts the typed model digests identically to the raw shape already published, and a live no-op re-run against the staging store reproduced the identical manifest digest. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Tests (327 -> 361): the precompute impl layer gets direct coverage — planner enumeration/presence, wave-worker guards (mismatched-key refusals, missing-artifact errors, budgetary-extras application), the determinism gate's frame comparator, the app wrappers' validate-in/dump-out boundary and local entrypoint wiring — plus seam locks: _build_simulation's predicate kwargs, publish-full-plan, the baseline_artifact export, the ArtifactOutcome/OUTCOME_* sync, and baseline_key's policy field. DRY/clean-code: one home each for uses_custom_data (predicate == baked-folder guard), resolve_data_folder (removes the /opt-vs-/tmp split-brain default in the precompute), baseline_artifact_filename, and the cohort resolution prologue; BundleReceipt renamed to BundleVersionIdentity (wire key unchanged); downloads land via a .partial temp name so a failed stream can't leave a truncated file exists() would trust; shared identity-seam stubs move to fixtures/identity_stubs.py. ensure() fixes: latch the artifact outcome (first outcome per request wins — the second ensure() self-cache-hit no longer rewrites a genuine miss to "hit", keeping the rollout metric honest) and actually replace the in-process cache entry after an incomplete-artifact recompute (LRUCache.add keeps the old value for existing keys, which made warm containers re-run the full model on every rich request instead of converging). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
anth-volk
marked this pull request as ready for review
July 23, 2026 22:31
This was referenced Jul 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #643
What
Phase 1 of the baseline artifact pipeline: everything needed to precompute the single-year US datasets (2026, 2027, 2025) and the 20 per-cohort national baselines per year into a content-addressed GCS store, and for the runtime to load those baselines instead of computing them. No behavior changes ship in this phase: with no artifacts in any image, every request takes exactly today's compute path (verified by the unchanged suite), and the store has no consumers in the deploy pipeline yet — the CI producer job and the image fetch layer come in the next phases.
Also rides along: the segmented-national poll cap drops 15s -> 8s (worst-case lag between a child finishing and the parent noticing).
How
artifact_keys— sha256 over canonical JSON of the full input closure: versions from the release bundle, the receipt's per-dataset content sha (closes the same-version data re-release hole thatforce_buildexists for), and the certificationdata_build_fingerprint. Baseline keys nest the dataset digest and carrycountry/region/scope_key(theScopingStrategy.cache_key), so UK or single-state artifacts later need no schema change. The deterministic Simulation idbl1-<digest16>doubles as the artifact filenameensure()already loads from. Golden tests turn any digest or upstreamcache_keyformat change into a reviewable diff.artifact_store— GCS client: write-once uploads (if_generation_match=0;PreconditionFailedis already-present success, the guard for concurrent beta/prod warmers), content-addressed manifests, last-writer-winsdeployed/<env>.jsonmarkers.google-cloud-storagebecomes an explicit dependency instead of a transitive ride-along via policyengine._build_simulationassigns the deterministic id when the baseline qualifies (current-law policy, macro scope, default data, national or region-group scope).ArtifactBaselineSimulation.ensure()validates any load (disk or LRU) againstresolve_entity_variablesand falls back torun()+save()on missing columns — cliff/LSR requests degrade to compute, never to wrong answers. Outcome exported as thebaseline_artifactattribute (hit/incomplete/miss).precompute_app— ephemeralmodal runwriter: plans against the store in-container (the bundle receipt the keys digest lives in the image, not on the deploying machine), computes only misses, builds baselines through the executor's own_resolve_region+_build_simulationon synthetic child requests (writer==reader by construction, asserted by a regression test), applies the same budgetary-pair extraseconomic_impact_analysiswould, runs a determinism gate (uploaded artifact vs independent fresh run, frame-by-frame), and printsMANIFEST_DIGEST=for the deploy pipeline.Tests
uv run pytest tests/in the executor project: 310 passed, 2 deselected. New:test_artifact_keys(30: golden digests, per-field sensitivity, upstream cache_key goldens, identity collection),test_artifact_store(9: write-once semantics, manifest round-trip, marker overwrites),test_baseline_artifacts(21: qualifying-predicate matrix, guard fallback/cache-replacement, class selection),test_precompute_app(10: function shapes incl. gcp secret and segment-worker sizing, miss-only planning, manifest filenames, writer==reader id contract, real-manifest stem check).ruff format/ruff checkclean on changed files.Not run:
make check's pyright step (binary not installed locally; CI lint runs ruff only). Not run yet: the manual stagingmodal runof the precompute app — it needs the artifact bucket provisioned (POLICYENGINE_ARTIFACT_BUCKET); that run is the phase gate before merge and will also produce the first real dataset-build and cohort-sim duration measurements.🤖 Generated with Claude Code