feat(autofix): Autofix introspection analytics#115891
Merged
Merged
Conversation
This adds an introspection after each autofix step that uses a LLM to judge if we should continue. For now this is behind a feature flag and will only emit analytics as we fine tune this.
Comment on lines
+422
to
+425
| prompt, | ||
| "introspection_decision_code_changes", | ||
| ) | ||
| except Exception: |
Contributor
There was a problem hiding this comment.
Bug: The introspection logic performs a synchronous, 30-second blocking poll within an RPC handler, which can cause worker timeouts and stall the entire autofix pipeline.
Severity: HIGH
Suggested Fix
The introspection logic should be executed asynchronously to avoid blocking the web worker and the main autofix pipeline. Move the call to run_introspection into a background task (e.g., using Sentry's task queue) so that it does not delay or prevent the trigger_autofix_agent call from executing promptly.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: src/sentry/seer/autofix/introspection.py#L422-L425
Potential issue: The `run_introspection` function, called from the
`call_on_completion_hook` RPC handler, initiates a synchronous poll for up to 30 seconds
by calling `client.get_run(..., blocking=True, poll_timeout=30)`. This blocks a web
worker for a long duration, risking worker timeouts and process kills. Because this
blocking analytics call occurs before the `trigger_autofix_agent` function is called to
continue the pipeline, a timeout or failure during introspection will prevent the
autofix process from proceeding to the next step. This turns an analytics feature into a
potential blocker for the entire autofix product.
Also affects:
src/sentry/seer/autofix/on_completion_hook.py:344~352
JoshFerge
approved these changes
May 21, 2026
Member
JoshFerge
left a comment
There was a problem hiding this comment.
makes sense to me! when ready we could add some evals for this as well!
JonasBa
pushed a commit
that referenced
this pull request
May 21, 2026
This adds an introspection after each autofix step that uses a LLM to judge if we should continue. For now this is behind a feature flag and will only emit analytics as we fine tune this.
natemoo-re
pushed a commit
that referenced
this pull request
May 21, 2026
This adds an introspection after each autofix step that uses a LLM to judge if we should continue. For now this is behind a feature flag and will only emit analytics as we fine tune this.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds an introspection after each autofix step that uses a LLM to judge if we should continue. For now this is behind a feature flag and will only emit analytics as we fine tune this.