Skip to content

feat(autofix): Break down not_eligible skip reasons for seer automation#111638

Merged
trevor-e merged 11 commits intomasterfrom
trevorelkins/aiml-2686-add-metrics-for-filtered-out-autofix-issues
Mar 26, 2026
Merged

feat(autofix): Break down not_eligible skip reasons for seer automation#111638
trevor-e merged 11 commits intomasterfrom
trevorelkins/aiml-2686-add-metrics-for-filtered-out-autofix-issues

Conversation

@trevor-e
Copy link
Member

@trevor-e trevor-e commented Mar 26, 2026

Builds on #111486 (already merged) which added initial seer.automation.filtered metrics and extracted filter logic into src/sentry/seer/autofix/trigger.py.

This PR adds two improvements:

1. Breaks down not_eligible into specific sub-reasons

The generic not_eligible skip reason is now 5 specific reasons:

  • not_eligible.issue_category_ineligible
  • not_eligible.gen_ai_feature_disabled
  • not_eligible.ai_features_hidden
  • not_eligible.scanner_not_enabled
  • not_eligible.no_budget

Moves is_issue_eligible_for_seer_automation and a new get_seer_automation_ineligibility_reason from utils.py into trigger.py to consolidate trigger decision logic.

2. Tracks below_occurrence_threshold as a skip reason

Issues with times_seen_with_pending < AUTOFIX_AUTOMATION_OCCURRENCE_THRESHOLD are now tracked as filtered from full automation. Summary generation (generate_issue_summary_only) still fires for these issues.

Refs AIML-2686

trevor-e and others added 10 commits March 24, 2026 18:37
…on issues

Extract filter logic from kick_off_seer_automation into two helper
functions that return skip reasons. This enables a single metrics.incr
and logger.info call per filtered issue, giving visibility into why
issues are being skipped (e.g. fixability too low, issue too old,
no connected repos, already triggered).

Refs AIML-2686
Co-Authored-By: Claude <noreply@anthropic.com>
Replace bare str return type with a SeerAutomationSkipReason Literal
union for type safety on skip reason values.

Refs AIML-2686
Co-Authored-By: Claude <noreply@anthropic.com>
Move SeerAutomationSkipReason, get_default_seer_automation_skip_reason,
and get_seat_based_seer_automation_skip_reason into a new
src/sentry/seer/autofix/trigger.py file. This makes the trigger
decision logic self-contained and easier to follow.

Refs AIML-2686
Co-Authored-By: Claude <noreply@anthropic.com>
Move lazy imports to top-level in trigger.py since there are no
circular dependency concerns. Pass the locks manager from
post_process.py rather than recreating it.

Refs AIML-2686
Co-Authored-By: Claude <noreply@anthropic.com>
Restore original comments in trigger.py and post_process.py. Use
`if skip_reason is not None` instead of `if skip_reason` for clarity.

Refs AIML-2686
Co-Authored-By: Claude <noreply@anthropic.com>
Revert to lazy imports in trigger.py to match original post_process.py
behavior and avoid mock patching issues. Move the >= 10 no-summary
rate limit check back to kick_off_seer_automation to avoid a TOCTOU
race from reading the cache twice.

Refs AIML-2686
Co-Authored-By: Claude <noreply@anthropic.com>
…NCE_THRESHOLD

Co-Authored-By: Claude <noreply@anthropic.com>
Add get_seer_automation_ineligibility_reason() which returns specific
reasons like not_eligible.issue_category_ineligible, not_eligible.no_budget,
etc. instead of a generic not_eligible. Move eligibility functions from
utils.py to trigger.py to consolidate trigger decision logic.

Refs AIML-2686
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
@linear-code
Copy link

linear-code bot commented Mar 26, 2026

@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Mar 26, 2026
…AutomationIneligibilityReason

Removes duplication between the two Literal types by composing
SeerAutomationSkipReason as a Union of its own literals and
SeerAutomationIneligibilityReason.

Refs AIML-2686
Co-Authored-By: Claude <noreply@anthropic.com>
@trevor-e trevor-e marked this pull request as ready for review March 26, 2026 16:19
@trevor-e trevor-e requested review from a team as code owners March 26, 2026 16:19
Copy link
Contributor

@chromy chromy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@trevor-e trevor-e merged commit 3c31979 into master Mar 26, 2026
70 checks passed
@trevor-e trevor-e deleted the trevorelkins/aiml-2686-add-metrics-for-filtered-out-autofix-issues branch March 26, 2026 17:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants