log: surface real failures in error.log#3127
Merged
Merged
Conversation
Promote 'unusual condition' messages from warning to error so error.log becomes useful as the first place to look when something goes wrong: - BaseModule.set_error_state: warning -> error (and critical when critical=True) -- the canonical 'this module gave up' signal. - DNS connectivity-failed message: warning -> error. - Dependency-install failures (pip / OS pkg / shell / Ansible / sudo-no-TTY): warning -> error. - Module setup hard-fail and module load failure in Scanner: warning -> error. - Scanner 'Aborting scan': hugewarning -> critical (still big-text in terminal AND now lands in error.log). - CLI: catch KeyboardInterrupt and any unhandled top-level Exception, log them via log.error + log.trace so the failure mode is recorded in error.log instead of dying silently to stderr. Demote noisy-but-handled wayback per-query failures (archive.org is chronically slow; the module retries cleanly) from warning to verbose. These were 2,400+ of the 2,600 warnings in a sample scan log.
Contributor
🚀 Performance Benchmark Report
|
scanner.setup-hard-failed and module set_error_state both now log at ERROR per this PR. update the cli test assertions accordingly.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## dev #3127 +/- ##
=====================================
- Coverage 90% 90% -0%
=====================================
Files 441 441
Lines 38743 38751 +8
=====================================
- Hits 34663 34662 -1
- Misses 4080 4089 +9 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
- kill_module: pass log_level="info" to set_error_state - KeyboardInterrupt in cli: warning instead of error - "Aborting scan": revert to hugewarning (triggered by Ctrl-C) set_error_state gains a log_level kwarg so callers can pick the severity. Genuine failures (setup hard-fail, API threshold, dep install, DNS dead) still log at ERROR.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #3123 (TRACE routing fix). Now that error.log is no longer drowning in TRACE noise, this audits what should be there and currently isn't — so that on a clean scan error.log is empty, and on a broken scan it's the one place you go.
Promoted from warning to error (currently surface as warnings in scan.log only)
BaseModule.set_error_stateDNSHelper'DNS queries are failing, please check your internet connection'DepsInstaller'Failed to install pip packages / OS packages / shell deps / Ansible tasks / sudo-no-TTY'Scanner'Setup hard-failed for {module}: {msg}'Scanner'Failed to load module {module_class}'Promoted from hugewarning to critical
Scanner.abort'Aborting scan'self.critical(..., trace=False)still renders big-text in the terminal but also lands in error.log.Added missing logging
cli.mainKeyboardInterruptcli.mainuncaught ExceptionBBOTError/KeyboardInterruptexception was falling through to Python's default handler — stderr only, never in error.log. Now caught with log.error + log.trace + non-zero exit.Demoted from warning to verbose (currently noise)
wayback.query'Error connecting to archive.org' / 'Error JSON-decoding archive.org response'Effect on the sample scan log
Before: 2,600 WARNINGs in scan.log, 0 ERRORs anywhere, 145K lines of TRACE noise in error.log.
After (estimated): ~160 WARNINGs, ~22 ERRORs in error.log surfacing actual broken modules + 'Aborting scan' + DNS failures + dep failures, ~0 lines on a clean scan.