fix(observability): capture AI-CLI stderr + surface exhausted provider failures to Sentry#1605
Merged
Conversation
…r failures to Sentry The claude/codex review CLIs intermittently failed with an opaque claude_code_exit_1 that was undiagnosable: defaultSpawn discarded stderr, and per-attempt provider failures logged at warn level, which the central Sentry forwarder (error/fatal only) skips. - defaultSpawn now captures the child stderr and threads it into the exit error, so the real cause (auth, rate limit, model-not-supported) reaches the logs. - runWorkersOpinion emits one ai_review_provider_exhausted log at error level once all models and attempts have thrown, so a genuine reviewer outage surfaces in Sentry while the noisy per-attempt retries stay at warn.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1605 +/- ##
=======================================
Coverage 95.50% 95.51%
=======================================
Files 204 204
Lines 22018 22030 +12
Branches 7957 7959 +2
=======================================
+ Hits 21029 21041 +12
Misses 413 413
Partials 576 576
🚀 New features to boost your workflow:
|
|
Warning 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 ⏸️ Gittensory review — held for maintainer review
⏸️ Held for maintainer review — Touches a guarded path — held for manual review
Nits — 1 non-blocking
Review context
Contributor next steps
Signal definitions
🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.
|
…errors The prior commit embedded raw claude/codex subprocess stderr into the thrown exit error. That stderr can echo the OAuth token we pass via env (or a key from a config the CLI reads), and the error flows to logs and the Sentry forwarder, whose scrubber only redacts secret-keyed fields, never free-text -- so a token inside the error string would leak. Add redactSecrets(): strip the caller's known secret values exactly, then well-known credential shapes (sk-/sk-ant-/sk-proj-, GitHub PAT/OAuth/server tokens, JWT, AWS key id), each word-boundary anchored so genuine diagnostics survive. Applied at both the claude (with the OAuth token) and codex throw sites before truncation.
Contributor
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
This was referenced Jun 27, 2026
Closed
JSONbored
added a commit
that referenced
this pull request
Jun 28, 2026
…the event slug (#1636) forwardStructuredLogToSentry titled each issue with only obj.event and buried the real failure in a 'log' context blob — so operators had to open each issue to learn what broke. Now: - title leads with the failure: "<event>: <message ?? error>" (e.g. orb_broker_unavailable: The operation was aborted due to timeout) - index filterable tags for the dimensions operators search/group by (repo, installationId, pull, pr, project, kind, deliveryId), present values only - fingerprint by event so recurrences collapse into one issue instead of fragmenting on the variable detail now in the title Pairs with the source-side instrumentation (#1605/#1619/#1625/#1627/#1628): those make failures reach Sentry; this makes each issue legible at a glance.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The self-host claude/codex review CLIs intermittently fail with an opaque
claude_code_exit_1/codex_exit_1, and provider outages never surface in Sentry. Two gaps:defaultSpawncaptured stdout but discarded stderr — where the CLI's real error (auth failure, rate limit, model-not-supported, OOM) actually lands. The exit code alone is undiagnosable.warn; the central Sentry forwarder only shipserror/fatal, so a genuine reviewer outage is invisible.Changes
src/selfhost/ai.ts—defaultSpawnnow captures the child's stderr and threads it into theclaude_code_exit_*/codex_exit_*error message (truncated to 500 chars). Thestderrfield is optional onSpawnFnso existing spawn stubs are unaffected.src/services/ai-review.ts—runWorkersOpinionemits oneai_review_provider_exhaustedlog at error level once all models × attempts have thrown (the reviewer is genuinely down). The noisy per-attempt retries stay atwarn. A model that runs but returns unparseable output is not an exhaustion (no provider error) and stays silent.Testing
defaultSpawncaptures a failing CLI's stderr (real subprocess) and surfaces it on the exit error.ai_review_provider_exhaustedfires at error level when every attempt throws; does not fire on ran-but-unparseable output.Advances #26 (comprehensive Sentry instrumentation — surface all failures).