fix(preflight): resolve API-key secrets via get_secret(), not stale env-var check (Sat SF DataPhase1 P0, facet 2)

[cipher813]

Context — second facet of the #241/#242 .env-deprecation regression

Surfaced during the 2026-05-16 Saturday SF recovery. #247 (AWS_REGION) worked: DataPhase1 cleared preflight and MorningEnrich completed end-to-end (Morning enrichment OK for 2026-05-15 ... in 1675s; polygon 913/921 + FRED 4/4 fetched fine via get_secret()) — so Friday's polygon-authoritative daily_closes are now collected and the original stale-data P0 is resolved.

But weekly_collector.py --phase 1 then aborted at preflight:

RuntimeError: Pre-flight: required env vars missing: ['FRED_API_KEY', 'POLYGON_API_KEY']

Root cause (reproduced, not speculative)

Every collector and both reachability probes in preflight.py itself already resolve these keys via get_secret() (SSM) — empirically proven by MorningEnrich succeeding with no env vars set. The only stale code is DataPreflight.run():

if self.mode == "phase1":
    self.check_env_vars("FRED_API_KEY", "POLYGON_API_KEY")      # stale os.environ assertion
elif self.mode == "phase2":
    self.check_env_vars("FMP_API_KEY", "FINNHUB_API_KEY", "EDGAR_IDENTITY")  # same latent bug

The env-deprecation arc migrated every consumer to get_secret() but missed this gate. MorningEnrich slipped through because its preflight only checks AWS_REGION; phase1/phase2 hard-fail on the stale gate. phase2 would have been the next failure.

Fix

• AWS_REGION stays an env-var check (plain boto3 region, not a secret — supplied by fix(spot): re-export AWS_REGION into spot shell — close #241 .env-removal regression (Sat SF DataPhase1 P0) #247's ENV_SOURCE).
• API keys go through a new _check_secrets() helper using get_secret(required=False) — identical <1s fail-fast intent and RuntimeError shape, sourced from SSM (with get_secret's env fallback) instead of os.environ.
• phase2 fixed in the same change (identical latent bug).

Test

Updated the two end-to-end tests that encoded the old env-var assumption to patch preflight.get_secret instead; added a renamed regression test (test_phase1_missing_polygon_secret_short_circuits) asserting the SSM-layer short-circuit still fires before HTTP/S3/ArcticDB. Full suite: 1050 passed, 1 skipped.

Recovery

After merge, re-trigger the Saturday SF. MorningEnrich's Friday fill from the recovery run is already persisted (ArcticDB + S3), so the re-run is safe; phase1 will now pass preflight and proceed.

🤖 Generated with Claude Code