feat(sf): Friday-PM shell_run dry-pass of the Saturday pipeline (spine; rule shipped disabled)

[cipher813]

Summary

Foundational spine of ROADMAP "Scheduled Friday-PM 'shell run' — automated full-fidelity preflight of the Saturday SF" (P1, added 2026-05-16). The prevention half of Saturday-SF reliability; the containment half (preflight-task-split) shipped 2026-05-16 in data #249/#250.

A scheduled Friday-PM dry/no-op pass of the entire Saturday SF turns a Saturday-fatal breakage from a Saturday-morning-after lost-week incident into a Friday-evening operator fix window (~11.5h before the real Sat 02:00 PT firing). Motivated by the recent multi-week Saturday-SF cascade history.

Design

Strict superset. shell_run absent/false ⇒ byte-identical to today's real Saturday run. Mirrors the existing skip_* / States.JsonMerge precedent exactly — no new mechanism invented. Only two existing edges change, each routed through a new Choice whose Default is the pre-spine target:

Edge	Pre-spine	Post-spine	Real Saturday run
InitializeInput.Next	CheckSkipMorningEnrich	CheckShellRun	CheckShellRun.Default → CheckSkipMorningEnrich (unchanged)
WaitForWeeklySubstrateHealthCheck.Next	NotifyComplete	CheckShellRunNotify	CheckShellRunNotify.Default → NotifyComplete (SUCCESS email untouched)

shell_run=true → ApplyShellRunDefaults (Pass): States.JsonMerge(<all 16 skip_*=true>, $, false) layers every skip flag = true under the execution input, so an explicit per-flag override still wins ({"shell_run":true,"skip_research":false} still runs Research). Every workload state already has a Choice-gated skip_*, so the whole workload no-ops via the existing skip mechanism.

Per-state dry-vs-skip inventory under shell_run

Spine = pure-skip (per-module "spots boot + smoke" dry paths are scoped follow-ons):

State(s)	Under shell_run	Mechanism
MorningEnrich, DataPhase1, RAGIngestion, RegimeSubstrate, RegimeRetrospectiveEval, Research, DataPhase2, EvalJudge(+RollingMean), RationaleClustering, ReplayConcordance, Counterfactual, PredictorTraining, DriftDetection, Backtester, Parity, Evaluator (16)	SKIPPED	existing skip_* Choice gate, force-true'd by ApplyShellRunDefaults
SaturdayHealthCheck, WeeklySubstrateHealthCheck	STILL RUNS (read-only — exactly the bootstrap/transport smoke the shell run wants)	no skip gate by design; shell_run-aware missing-Friday-bar tolerance = ROADMAP owed-work item 5 (follow-on)
(success notify)	NotifyShellRunComplete	shell-run-tagged Subject, reuses the exact NotifyComplete SNS substrate

Friday cron + justification

cron(30 21 ? * FRI *) = 21:30 UTC Fri = 14:30 PT (PDT, dominant season) / 13:30 PT (PST). Chosen after the Friday EOD SF (~1:25 PT) so the shell run never collides with PostMarketData/EODReconcile/StopTradingInstance on the trading instance, and ~11.5h before the real Sat 09:00 UTC firing — a red report Friday evening gives a full operator fix window. shell_run mode fetches nothing, so the Saturday "polygon T+1 must have settled" timing constraint is moot here.

Operator enable step

The rule ships State: DISABLED — zero-risk merge. Additive observability, not a backstop (the "fail loud, no backstop" design decision stands). Brian enables deliberately:

aws events enable-rule --name alpha-engine-friday-shell-run --region us-east-1

It targets the same alpha-engine-saturday-pipeline SF (NOT a parallel SF) with {"shell_run": true}, same EventBridgeSfnRoleArn. The existing states:StartExecution grant is SF-ARN-scoped (infrastructure/iam/alpha-engine-eventbridge-sfn-role.json) so no IAM change is needed.

Consolidated-notify decision

Shell-run SUCCESS reuses the existing NotifyComplete SNS pattern with a SHELL RUN-tagged Subject (zero new infra). A shell-run FAILURE reuses the unchanged HandleFailure — its 20 inbound error edges were deliberately not re-pointed (high churn, zero added operator value, would perturb the real Saturday failure path's risk surface; the FAILED alert's Friday execution timestamp/ID is the actionable signal). The richer per-state pass/fail report (ROADMAP design point 5) is a scoped follow-on.

Scoped per-module follow-on PRs (NOT in this spine)

Convert "skipped" → "spots boot + smoke":

Repo	State	Dry mode needed
alpha-engine-data	DataPhase1 / MorningEnrich	spot_data_weekly.sh --preflight-only (preflight + universe-freshness scan, no polygon/FMP writes)
alpha-engine-data	RAGIngestion	spot_data_weekly.sh --rag-only --preflight-only (corpus reachability + secrets, no SEC/embedding writes)
alpha-engine-predictor	PredictorTraining	spot_train.sh --preflight-only (load + WF-gate-shape check, NO predictor/weights/ promotion)
alpha-engine-backtester	Backtester / Parity / Evaluator	spot_backtest.sh --mode=smoke + simulate-dry, NO config/*.json auto-apply (freeze_evaluator pattern)
alpha-engine-data	SaturdayHealthCheck / WeeklySubstrateHealthCheck	shell_run-aware missing-Friday-bar tolerance (ROADMAP owed-work item 5)

Research / predictor-inference / executor already have --dry-run/--simulate; wiring those into the SF states is part of the per-state follow-ons above.

Tests

tests/test_sf_friday_shell_run_wiring.py — 23 cases: strict-superset edges, JsonMerge user-input-wins order, every skip-gate covered by the defaults blob (drift guard), full happy-path traversal for shell_run true vs absent, Friday rule DISABLED + same-SF + shell_run=true + cron. Updated two pre-spine wiring tests (morning_enrich_split, substrate_check) to assert through the new gates while pinning Default == pre-spine target. Full suite: 1242 passed, 1 skipped (pre-existing, unrelated). No new pip deps. No secrets.

🤖 Generated with Claude Code