Skip to content

fix(spans): Avoid unpack() crashes when merging sets#113442

Merged
untitaker merged 1 commit intomasterfrom
sadd-batched
Apr 20, 2026
Merged

fix(spans): Avoid unpack() crashes when merging sets#113442
untitaker merged 1 commit intomasterfrom
sadd-batched

Conversation

@untitaker
Copy link
Copy Markdown
Member

unpack() can crash if it gets more than 200 arguments.

ref INC-2130

unpack() can crash if it gets more than 200 arguments.

ref INC-2130
@untitaker untitaker marked this pull request as ready for review April 20, 2026 16:03
@untitaker untitaker requested review from a team as code owners April 20, 2026 16:03
@github-actions github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label Apr 20, 2026
Comment on lines +60 to 70
-- at dest_key can be massive. if it is, SUNIONSTORE will copy the
-- entire target set again, which appears to be _worse_ than
-- copying the source set into lua memory and out again.
redis.call("sadd", dest_key, unpack(members))
end
until cursor == "0"
end

local num_spans = ARGV[1]
local parent_span_id = ARGV[2]
local has_root_span = ARGV[3] == "true"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The unpack(hset_args) call in the Lua script may exceed the argument limit if num_spans is large, as there is no explicit chunking for the hset operation.
Severity: HIGH

Suggested Fix

Apply a similar batching mechanism to the hset call as was done for the sadd calls in the same script. This involves iterating over the arguments in chunks to avoid exceeding the Lua stack limit when calling unpack.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: src/sentry/scripts/spans/add-buffer.lua#L43-L70

Potential issue: The Lua script at `src/sentry/scripts/spans/add-buffer.lua` uses
`redis.call("hset", ..., unpack(hset_args))`. The `unpack` function has a stack limit of
around 8000 arguments. Since `hset_args` contains two entries per span, if the number of
spans exceeds approximately 3999, the call will fail with a stack overflow, causing a
runtime crash. While other calls like `sadd` were updated to handle large numbers of
arguments by batching, this `hset` call was not, despite the `max_spans_per_evalsha`
setting defaulting to an unlimited value (0). This appears to be an oversight.

Also affects:

  • src/sentry/scripts/spans/add-buffer.lua:124

Did we get this right? 👍 / 👎 to inform future reviews.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we limit EVALSHA on the caller side

@untitaker untitaker merged commit c24b146 into master Apr 20, 2026
51 checks passed
@untitaker untitaker deleted the sadd-batched branch April 20, 2026 16:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants