fix(detection): reduce trace selection batch size from 50 to 5#114372
Merged
roggenkemper merged 6 commits intomasterfrom Apr 30, 2026
Merged
fix(detection): reduce trace selection batch size from 50 to 5#114372roggenkemper merged 6 commits intomasterfrom
roggenkemper merged 6 commits intomasterfrom
Conversation
The detection pipeline queries top N transactions per org, then runs 1 Snuba query per transaction to sample a trace — but only sends 1 trace to Seer. At TRANSACTION_BATCH_SIZE=50, that's ~52 Snuba queries to select 1 trace. Reducing to 5 cuts selection queries by ~90% while still providing enough candidates after span-count and dedup filtering. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
github-actions
Bot
added
the
Scope: Backend
…e queries Instead of reducing TRANSACTION_BATCH_SIZE, keep querying top 50 transactions for diversity but shuffle and sample 5 before the per-transaction trace sampling loop. This cuts ~45 Snuba queries per org execution while maintaining diverse transaction coverage. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Transactions are already shuffled in trace_data.py before sampling, so the shuffle of evidence_traces here was redundant. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Only 1 trace is sent to Seer per execution, and neither span count filtering nor dedup replaces filtered traces with alternatives, so extra candidates don't add value. Cuts per-transaction trace queries from 5 to 1 per org. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
With TRACE_SAMPLE_SIZE=1 from a shuffle of 50 transactions, the odds of picking the same transaction twice across runs are ~2%. Combined with longer execution cycles, the Redis-based dedup adds complexity without meaningful value. Removes _get_unprocessed_traces, mark_traces_as_processed, and TRACE_PROCESSING_TTL_SECONDS. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Span count filtering (20-500 spans) can reject candidates, so we need a few to have fallbacks. 5 gives enough margin while still cutting queries from 50 to 5. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
cvxluo
approved these changes
Apr 30, 2026
cleptric
pushed a commit
that referenced
this pull request
May 5, 2026
Reduces `TRANSACTION_BATCH_SIZE` from 50 to 5 in the AI issue detection trace selection pipeline. We used to take 10 traces from the 50, now that we are only taking 1, we can lower. also removes redis deduplication that is less relevant given the longer cycles
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reduces
TRANSACTION_BATCH_SIZEfrom 50 to 5 in the AI issue detection trace selection pipeline. We used to take 10 traces from the 50, now that we are only taking 1 we can lower. also removes redis deduplication that is less relevant given the longer cycles