Skip to content

fix(freshness-monitor): deploy.sh preflight test step now sees runtime deps#337

Merged
cipher813 merged 1 commit into
mainfrom
fix/freshness-monitor-deploy-test-deps
May 27, 2026
Merged

fix(freshness-monitor): deploy.sh preflight test step now sees runtime deps#337
cipher813 merged 1 commit into
mainfrom
fix/freshness-monitor-deploy-test-deps

Conversation

@cipher813
Copy link
Copy Markdown
Owner

Summary

Fixes the ModuleNotFoundError: No module named 'yaml' failure when running bash infrastructure/lambdas/freshness-monitor/deploy.sh against a Python environment that doesn't have the Lambda's runtime deps pre-installed.

Root cause

PR #335 placed the preflight test step (python3 -m pytest test_handler.py) BEFORE the pip-install step. The test imports index.py which imports yaml + boto3 + alpha_engine_lib.{alerts,artifact_freshness} at module load — none of which are guaranteed in the caller's bare python. Same issue applied to the registry validator (also imports yaml).

The pattern was mirrored from sf-telegram-notifier/deploy.sh, but that Lambda's test_handler.py STUBS all its 3rd-party imports via sys.modules (only pytest is a real dep). The freshness-monitor's test exercises real lib substrate (ArtifactSpec, CheckResult, check_freshness) so stubbing those defeats the test's purpose — installing the deps is the right answer.

Fix

Restructure step ordering so deps install before consumers run:

Step Before After
0a syntax-check syntax-check (unchanged)
0b registry validator (fails — no yaml) path-check only
0c preflight tests (fails — no yaml) — (moved)
1 pip install runtime deps → $PKG pip install runtime deps → $PKG
1a registry validator with PYTHONPATH=$PKG
1b install pytest$TEST_DEPS; run handler tests with PYTHONPATH=$PKG:$TEST_DEPS
1c zip Lambda zip Lambda (unchanged)

The $TEST_DEPS separate dir keeps pytest + its transitive deps out of the final Lambda zip — only requirements.txt deps end up in the deployable artifact. Both temp dirs cleaned via the existing trap.

Verified

End-to-end dry-run on the operator machine (homebrew python3):

index.py syntax OK
Installing runtime deps into /var/folders/.../tmp.pkeQqBGGoA (pip install -t)...
Validating registry locally before upload...
✅ ARTIFACT_REGISTRY.yaml validated: 48 artifacts + 27 grandfathered paths
Installing pytest into /var/folders/.../tmp.wKKWUBNrc6...
Running handler unit tests (PYTHONPATH=...:...)...
............                                                             [100%]
12 passed in 0.08s
Packaged function.zip (22498727 bytes)
✓ Code deployed.
✓ Registry uploaded.

Why this matters

deploy.sh is the only path operators have to ship this Lambda; if the preflight fails on Brian's machine the artifact-freshness-monitor arc's Phase 6 deploy is blocked. This is a one-character-deep bug class (if [[ -f test_handler.py ]]; then python3 -m pytest assumes deps available) that bit because PR #335 mirrored the sf-telegram-notifier deploy.sh pattern without noting the test-stubbing difference.

Test plan

  • bash infrastructure/lambdas/freshness-monitor/deploy.sh --dry-run — clean end-to-end
  • pytest infrastructure/lambdas/freshness-monitor/test_handler.py (in lib venv) — 12 passing
  • Post-merge: Brian re-runs --bootstrap and instantiates the Lambda

🤖 Generated with Claude Code

…e deps

Problem: PR #335 placed the preflight test step (`python3 -m pytest
test_handler.py`) BEFORE the pip install step. The test imports
index.py which imports yaml + boto3 + alpha_engine_lib.{alerts,
artifact_freshness} at module load — none of which are guaranteed
in the caller's bare python. Brian's run hit ModuleNotFoundError
on yaml because his shell python lacked pyyaml. Same root cause
applied to the registry validator (also imports yaml).

Fix: restructure step ordering so deps install before consumers
run:

  0a. Syntax-check handler (no imports — works on bare python)
  0b. Verify ae-config clone present (path check only)
  1.  Install runtime deps into $PKG via pip --target
  1a. Validate registry with PYTHONPATH=$PKG (yaml available)
  1b. Install pytest into separate $TEST_DEPS dir; run handler
      tests with PYTHONPATH=$PKG:$TEST_DEPS (all deps available;
      pytest NOT bundled into Lambda zip)
  1c. Copy handler + zip Lambda package (unchanged)

The $TEST_DEPS separate dir keeps pytest + its transitive deps out
of the final Lambda zip — only runtime deps from requirements.txt
end up in the deployable artifact. Both temp dirs cleaned via the
existing trap.

End-to-end dry-run verified:
  index.py syntax OK
  ✅ ARTIFACT_REGISTRY.yaml validated: 48 artifacts + 27 grandfathered paths
  12 passed in 0.08s
  Packaged function.zip (22498727 bytes)

Why this matters: deploy.sh is the only path operators have to ship
this Lambda; if the preflight fails on Brian's machine the entire
arc's Phase 6 deploy is blocked. This is a one-character-deep bug
class (`if [[ -f test_handler.py ]]; then python3 -m pytest`
assumes deps available) that bit because PR #335 mirrored the
sf-telegram-notifier deploy.sh pattern without noting that
sf-telegram-notifier's test stubs all its 3rd-party imports via
sys.modules whereas this Lambda's test exercises real lib substrate.

Composes with the Phase 6 OBSERVE-mode deploy that's the only
remaining item on the artifact-freshness-monitor arc.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cipher813 cipher813 merged commit ad61b92 into main May 27, 2026
1 check passed
@cipher813 cipher813 deleted the fix/freshness-monitor-deploy-test-deps branch May 27, 2026 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant