fix(daily_closes): scrub FRED api_key from error logs + retry FRED 5xx#220
Merged
Conversation
requests.exceptions.HTTPError.str() embeds the full request URL —
including ``api_key=<credential>`` — in its message. The existing
``_fetch_fred_closes`` except-block logged that raw via
``logger.warning("FRED fetch failed for %s (%s): %s", t, sid, e)`` so
any FRED 5xx in production MorningEnrich / EOD dumped the credential
to CloudWatch.
Surfaced 2026-05-12 during the post-merge repair run for PR #219 — a
transient FRED 500 on VXVCLS during the operator step leaked the key
to the conversation transcript. The FRED key is rotated separately;
this commit closes the leak class.
Changes:
- ``_scrub_api_key(msg) -> str`` helper in ``collectors/daily_closes.py``
masks the ``api_key=...`` querystring fragment with ``api_key=***``
- ``_fetch_fred_closes`` warn-log routes the exception through the
scrubber
- Repair script's ``_fetch_fred_range`` already added retry+scrub in
the cherry-pick from the live-debug session; this commit consolidates
by importing the shared ``_scrub_api_key`` from ``daily_closes`` so
the regex doesn't drift across files
Tests (+7, suite 792 → 799):
- ``_scrub_api_key`` masks URL-embedded keys, handles exception objects
directly, passthroughs when no key present, terminates at ``&``
- ``_fetch_fred_closes`` warn-log scrub regression
- ``_fetch_fred_range`` retry-warn + final-error scrub regressions
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes a latent credential-leak class in the FRED fetcher's error-log path.
requests.exceptions.HTTPError.str()embeds the full request URL — includingapi_key=<credential>— and_fetch_fred_closeslogged it verbatim vialogger.warning("... %s", e). Any FRED 5xx in production MorningEnrich / EOD would dump the key to CloudWatch logs.Surfaced 2026-05-12 during the post-merge operator step for PR #219 — a transient FRED 500 on VXVCLS during the repair-script run leaked the key to the conversation transcript. The FRED key is being rotated separately; this PR closes the underlying leak.
What changed
collectors/daily_closes.py— new_scrub_api_key(msg) -> strhelper masks theapi_key=...querystring fragment withapi_key=***._fetch_fred_closes's warn-log routes the exception through the scrubber.collectors/daily_closes_fred_repair.py— adds retry (3 attempts, 3s/6s backoff) on transient FRED 5xx + imports the shared_scrub_api_keyfromdaily_closes(regex defined once, no drift across files).tests/test_fred_api_key_scrub.py(new, +7 tests) — pins the scrubbing contract on_scrub_api_keydirectly + on_fetch_fred_closeswarn path + on_fetch_fred_rangeretry-warn + final-error paths.Suite 792 → 799.
Test plan
pytest -qclean (799 passed, 1 skipped pre-existing)api_key=***not the raw key..envlocally +alpha-engine-config/data/.env+ the AWS SSM/alpha-engine/FRED_API_KEYparameter (or wherever the Lambda env loader reads from). Rotation is independent of this PR — the PR closes the future leak surface.🤖 Generated with Claude Code