fix(integrations): treat 5xx SCM responses as halts, not SLO failures - #120733
Draft
billyvg wants to merge 1 commit into
Draft
fix(integrations): treat 5xx SCM responses as halts, not SLO failures#120733billyvg wants to merge 1 commit into
billyvg wants to merge 1 commit into
Conversation
When a third-party SCM provider (e.g. Bitbucket) returns a 5xx response during `check_file`, the ApiError previously propagated out of the IntegrationEventLifecycle context manager, triggering `record_failure` with `create_issue=True`. This caused spurious Sentry issues for conditions that are transient and outside our control. Fix: add a 5xx guard in `RepositoryIntegration.check_file`'s `except ApiError` handler that calls `lifecycle.record_halt(e)` and returns `None`, consistent with how 404/400 responses are already handled. Fixes https://sentry.io/organizations/sentry/issues/7549081279/ Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N4onAVtnECuj7KrXLyavhg
Contributor
Backend Test FailuresFailures on
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes a noisy instrumentation issue where transient third-party 5xx responses during stacktrace link resolution were creating Sentry issues via
record_failure(create_issue=True).Sentry issue: https://sentry.io/organizations/sentry/issues/7549081279/
Dispatched by: automated triage routine (Claude session https://claude.ai/code/session_01N4onAVtnECuj7KrXLyavhg)
Root-cause analysis
When Bitbucket's API returns a 503 Service Unavailable during a
check_filecall (stacktrace link resolution), the error path was:ProjectStacktraceLinkEndpoint.get→get_stacktrace_config→get_link(stacktrace_link.py)get_linkcallsinstall.get_stacktrace_link(...), catching onlyApiErrorwithcode == 403and re-raising everything elseBitbucketIntegration.get_stacktrace_link→RepositoryIntegration.check_fileinsource_code_management/repository.pycheck_file,client.check_file()raisesApiError(code=503)from the HTTP responseexcept ApiError as e:handler only halted on codes404and400; theelse: raisebranch re-raised 5xx errorswith self.record_event(...).capture() as lifecycle:blockIntegrationEventLifecycle.__exit__calledself.record_failure(exc_value, create_issue=True)— creating a Sentry issue for an external service being temporarily unavailableThis produced false-positive SLO failures for a condition entirely outside Sentry's control.
What the fix does
In
RepositoryIntegration.check_file(src/sentry/integrations/source_code_management/repository.py), added a 5xx guard inside theexcept ApiError as e:handler, immediately after the existing404/400halt:record_halt(unlikerecord_failure) does not callsentry_sdk.capture_exceptionand does not create a Sentry issue by default. This matches how 404/400 responses are already handled — they represent expected or benign conditions that don't warrant an issue.Returning
Nonecausesget_stacktrace_linkto returnNonenormally (success path), so neither the innerCHECK_FILEnor the outerGET_STACKTRACE_LINKlifecycle records a failure.What was considered and ruled out
get_link(stacktrace_link.py): That function only sees the exception after it has already escaped both lifecycle context managers (triggering tworecord_failurecalls). Fixing there would suppress the duplicate re-raise but would not prevent the instrumentation noise at the source.get_stacktrace_link(repository.py): Similar problem — by the time the exception reachesget_stacktrace_link's outertry/except, theCHECK_FILElifecycle has already recorded a failure. The fix needs to be at the point where theApiErroris first caught inside the lifecycle.create_issue=Trueinrecord_failure, not log level. Changing a log level would not prevent the Sentry issue from being created.ApiRetryErrorfor non-GitLab providers:ApiRetryError(code=503, fromrequests.adapters.RetryError) is caught in a separateexceptclause beforeexcept ApiError. The existing TODO comment indicates that behavior is intentionally deferred. This fix only addresses plainApiError5xx responses from HTTP responses, which is the path the reported issue takes.Tests
Added two tests to
tests/sentry/integrations/bitbucket/test_client.py:test_check_file_5xx_returns_none_and_records_halt: verifies that a 503 from Bitbucket causescheck_fileto returnNoneand recordsHALTED(notFAILURE) in the lifecycletest_get_stacktrace_link_5xx_returns_none: verifies end-to-end that noFAILUREoutcome is recorded when the SCM provider is temporarily unavailableLegal Boilerplate
Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. and is gonna need some rights from me in order to utilize my contributions in this here PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.
Generated by Claude Code