Analyze for known errors when an action fails#204
Merged
Conversation
to a shared location where we can use it in doctor
rubberduck203
requested review from
meaganewaller,
noizwaves and
smacfarlane
and removed request for
noizwaves
meaganewaller
approved these changes
Jun 17, 2025
rubberduck203
commented
Jun 19, 2025
So we can pass the output from action fixes to analyze_known_errors without allocating a giant string
rubberduck203
added a commit
that referenced
this pull request
Apr 15, 2026
## Why \`scope-intercept\` is used as a shebang interpreter for setup scripts. When the wrapped command fails, it already detected known errors and showed help text — but it never ran fixes or retried the script. This meant engineers still had to manually apply the fix and re-run the script themselves. \`scope doctor\` gained self-healing behaviour in #204: when a fix succeeds, the action is automatically retried. This PR extends the same behaviour to \`scope-intercept\`. ## What changed **`src/shared/mod.rs`** — Changed `pub(crate) mod analyze` to `pub mod analyze` so the shared fix-and-analyze infrastructure is accessible from the binary crates. **`src/shared/analyze/mod.rs`** — Added `yolo: bool` to `process_lines` and `prompt_and_run_fix`. When `true`, known-error fix prompts are auto-approved without requiring a TTY. This is the shared fix-prompt path used by both `scope-intercept` and `scope doctor`. **`src/bin/scope-intercept.rs`** — Added `--yolo` / `-y` flag. On command failure: 1. Feed captured output through `analyze::process_lines` to match known errors, prompt, and run fixes. 2. If the fix succeeds, re-run the entire original command. 3. If the retry succeeds, return exit 0. If it fails, fall through to the bug-report flow. 4. Extracted `offer_bug_report()` to avoid duplicating that logic across the original-failure and retry-failure paths. **`src/doctor/check.rs`** / **`src/doctor/commands/run.rs`** — Bug fix discovered while adding `--yolo` to scope-intercept: `scope doctor --yolo` was not auto-approving known-error fix prompts that fire during check analysis (`analyze_known_errors`), even though it correctly auto-approved the check's own fix prompts. The `yolo` flag was never threaded into `DefaultDoctorActionRun`. Fixed by adding `yolo: bool` to the struct and passing `args.yolo` at construction. **`src/analyze/cli.rs`** — Updated `process_lines` call sites to pass `yolo: false`; added a comment explaining why `scope analyze` is always interactive and doesn't expose `--yolo`. ## Tests **7 E2E integration tests** (`tests/scope_intercept.rs`): - Fix succeeds → retry succeeds (direct command and via script / shebang path) - Fix succeeds → retry still fails (fix resolves one error, second remains) - Known error found, no fix available - No TTY without `--yolo` → fix denied, command fails - Command succeeds on first try (no analysis triggered) - No known errors match → original exit code preserved **4 unit tests** (`src/doctor/check.rs`, `analyze_known_errors_spec`): - `yolo: true` auto-approves the known-error fix - `yolo: false` without a TTY returns `KnownErrorFoundUserDenied` - Output not matching any pattern returns `NoKnownErrorsFound` - Pattern match with no fix configured returns `KnownErrorFoundNoFixFound` 🤖 Generated with [Claude Code](https://claude.com/claude-code) [LDE-463]: https://gustohq.atlassian.net/browse/LDE-463?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements #195
AI Summary
This pull request introduces enhancements to error handling and self-healing mechanisms in the
scopeproject, alongside updates to YAML configuration files for defining known errors and their fixes. The most significant changes include adding support for self-healing during action fixes, refactoring error analysis logic, and updating known error definitions.DefaultDoctorActionRunto analyze known errors when fixes fail, display help text, and optionally re-run fixes if a known error's fix succeeds (scope/src/doctor/check.rs).run_fixes_with_self_healingto handle this process, and refactored the action result logic to streamline error handling and verification (scope/src/doctor/check.rs). [1] [2]process_lines,report_result) to theshared::analyzemodule so the known error logic can be reused. (scope/src/analyze/cli.rs). [1] [2] [3]doctor-group-auto-fix.yamlconfiguration file to demonstrate auto-fixing workflows for known errors (examples/.scope/auto-fix/doctor-group-auto-fix.yaml).action-failed,verification-failed) and their fixes, enabling automatic detection and resolution of these errors (examples/.scope/auto-fix/known-error-action-failed.yaml,examples/.scope/auto-fix/known-error-verification-failed.yaml). [1] [2]error-existsinknown-error.yamlto use a stricter regex pattern for error detection so it doesn't trigger when running the new example (examples/.scope/known-error.yaml).is_successmethod toActionRunStatusfor better readability and encapsulation (scope/src/doctor/check.rs).scope/src/doctor/check.rs).Note on testing
The unit testing leaves a little to be desired.
The
analyzecode had no existing unit tests around it.https://github.com/oscope-dev/scope/blob/5a0de9051f39396406e7dc0542a19cd918a60884/scope/src/analyze/cli.rs
And the prompting in the middle of the processing makes it difficult to get this under test.
https://github.com/oscope-dev/scope/blob/5a0de9051f39396406e7dc0542a19cd918a60884/scope/src/analyze/cli.rs#L125-L129
Further refactoring needs to happen in order to get this new functionality properly under test.
We need to do something like we did for action runs
https://github.com/oscope-dev/scope/blob/5a0de9051f39396406e7dc0542a19cd918a60884/scope/src/doctor/check.rs#L170
passing the prompt as a callback.
The tests for
check.rsare also just kind of clunky and it's hard to setup these tests in general.The framework that is in place for the
checktests is now being stretched beyond it's original design.I think part of the problem is all of the tests were written as an end to end test of the
run_actionfunction and if we extract more functions and test at a lower level this will get easier, but will be more work and a bigger change.I plan on doing some of this refactoring and testing as I have moments to do so, but for now I tested this new feature manually via the new example.