Skip to content

Strip console warnings before classifying infra failures#2569

Merged
hiroshinishio merged 1 commit intomainfrom
wes
Apr 21, 2026
Merged

Strip console warnings before classifying infra failures#2569
hiroshinishio merged 1 commit intomainfrom
wes

Conversation

@hiroshinishio
Copy link
Copy Markdown
Collaborator

@hiroshinishio hiroshinishio commented Apr 21, 2026

Summary

  • detect_infra_failure was substring-matching AccessDeniedException in app-level console.warn output (SSM fetch fallback), sending PRs with real Jest failures down the infra-retry branch. Three empty-commit retries later, GA gave up without ever invoking the LLM fix path.
  • Fix: call strip_jest_noise first so console.warn/console.log blocks (including the orphaned AWS stack traces and closing braces) are removed before pattern scanning. The classifier's single responsibility — match infra patterns in cleaned log — is preserved.
  • Regression test loads the real 396KB CircleCI log captured from Foxquilt/foxden-rating-quoting-backend PR 714 and asserts it now classifies as None.

Notes

  • CLAUDE.md: documented that .env and .venv are already sourced before Claude starts, plus the python -m pytest ... invocation (so top-level imports like from constants.models ... resolve without PYTHONPATH).
  • .gitignore: .claude/scheduled_tasks.lock.

Social Media Post (GitAuto)

Infrastructure-failure classifier no longer trips on app-level console warnings

  • Console.warn/log blocks stripped before scanning, so app warnings (failed SSM fetches, etc.) can't false-positive the classifier
  • Regression test uses a real captured CircleCI log that previously burned three empty-commit retries
  • PRs with real test failures now reach the LLM fix path on the first run

Social Media Post (Wes)

Spent the morning on a classifier that confidently called a code bug "infra" because the app logged an AccessDenied warning during test setup. Three empty-commit retries later, nothing fixed. Strip the console blocks before scanning and the same log classifies correctly. One responsibility per function, clean input in.

detect_infra_failure was substring-matching "AccessDeniedException" in
app-level console.warn output (SSM fetch fallback), routing real Jest
test failures down the CI-retry path. Call strip_jest_noise first so
app warnings can no longer false-positive the classifier.

Regression test loads the real CircleCI log from Foxquilt PR 714 that
triggered the bug and asserts it now classifies as None. Also documents
the python -m pytest invocation and ignores .claude/scheduled_tasks.lock.
@hiroshinishio hiroshinishio self-assigned this Apr 21, 2026
@hiroshinishio hiroshinishio merged commit 77c82bc into main Apr 21, 2026
1 check passed
@hiroshinishio hiroshinishio deleted the wes branch April 21, 2026 22:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant