fix(data-lambda): read FMP+FINNHUB via get_secret() — unblock post-PR-9f deploy#242
Merged
Merged
Conversation
…viron
PR 9f deleted the ssm_secrets.load_secrets() shim that populated
os.environ from SSM at Lambda cold-start. Handler still gated on
os.environ.get(name) for FMP_API_KEY + FINNHUB_API_KEY, so the
post-PR-9f deploy canary now fails:
WARNING: Canary returned 'Missing required env vars: FINNHUB_API_KEY'
Only FINNHUB surfaces because FMP_API_KEY happens to be persisted in
the live Lambda env config from a legacy .env-deploy; FINNHUB only
ever made it to os.environ via the shim. Switch the handler's pre-flight
check to alpha_engine_lib.secrets.get_secret(), matching the pattern
already in use at every other secret-consumer site in this repo
(collectors/fundamentals.py:209, collectors/finnhub_client.py:63,
collectors/alternative.py:469,757,836).
Test gap note: tests/test_no_secret_environ_reads.py regex only
matches string-literal forms (os.environ.get("NAME")), so the
variable-based loop pattern (os.environ.get(name)) slipped past.
Worth tightening in a follow-up but not strictly required — the
fix here removes the only remaining secret read of that shape in
the repo.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
cipher813
added a commit
that referenced
this pull request
May 16, 2026
…nv-var check (#248) Second facet of the #241/#242 .env-deprecation regression, surfaced by the 2026-05-16 Saturday SF recovery run: AWS_REGION fix (#247) let DataPhase1 clear preflight and MorningEnrich completed (polygon 913/921 + FRED 4/4 fetched fine via get_secret()), but `weekly_collector.py --phase 1` then aborted at preflight: RuntimeError: Pre-flight: required env vars missing: ['FRED_API_KEY', 'POLYGON_API_KEY'] Every collector AND both reachability probes in this file already resolve these keys via get_secret() (SSM). The only stale code was DataPreflight.run()'s `check_env_vars("FRED_API_KEY","POLYGON_API_KEY")` (and the phase2 FMP/FINNHUB/EDGAR equivalent) — an os.environ assertion the env-deprecation arc migrated every consumer away from but missed here. MorningEnrich slipped through because its preflight only checks AWS_REGION; phase1/phase2 hard-failed on the stale gate. Fix: AWS_REGION stays an env-var check (plain boto3 region, not a secret); the API keys now go through a new `_check_secrets()` helper that calls get_secret(required=False) — same <1s fail-fast intent, same RuntimeError shape, sourced from SSM (with get_secret's env fallback) instead of os.environ. phase2 had the identical latent bug and is fixed in the same change. Tests updated to the get_secret() reality (patch preflight.get_secret rather than os.environ); full suite 1050 passed, 1 skipped. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
lambda/handler.pypre-flight gated onos.environ.get(name)forFMP_API_KEY+FINNHUB_API_KEY. After PR feat(data): delete ssm_secrets + .env + push-secrets.sh + spot .env sourcing (PR 9f) #241 (PR 9f of the.env→ SSM arc) deleted thessm_secrets.load_secrets()bulk-load shim, nothing populatesos.environat cold-start anymore — so the post-merge deploy canary fails withMissing required env vars: FINNHUB_API_KEYand rolls back.FINNHUB_API_KEYsurfaces (not both) becauseFMP_API_KEYhappens to be persisted in the live Lambda env config from a legacy.env-deploy;FINNHUB_API_KEYwas only ever loaded via the shim.alpha_engine_lib.secrets.get_secret()— matching the pattern already in use at every other secret-consumer site in this repo (collectors/fundamentals.py:209,collectors/finnhub_client.py:63,collectors/alternative.py:469,:757,:836).Why this slipped past CI
tests/test_no_secret_environ_reads.pyregex only matches string-literal forms (os.environ.get("NAME")). The variable-based loop pattern (os.environ.get(name)wherenameiterates a tuple of pinned secret names) doesn't match. Worth tightening the AST in a follow-up, but this fix removes the only remaining read of that shape in the repo.Test plan
pytest tests/test_no_secret_environ_reads.py tests/test_preflight.py tests/test_fundamentals_finnhub.py -q→ 50 passedimport lambda/handler.pysmoke-loads cleanly under.venvalpha-engine-data-rolehasssm:GetParameteron/alpha-engine/*(inlinealpha-engine-ssm-readpolicy)/alpha-engine/FMP_API_KEY+/alpha-engine/FINNHUB_API_KEYboth exist.github/workflows/deploy.yml) — should pass now🤖 Generated with Claude Code