Skip to content

process-spans: TypeError comparing evalsha_latency_ms (bytes) with float when using Dragonfly as Redis #116116

@krisztiansala

Description

@krisztiansala

Environment

Self-hosted Sentry via sentry-kubernetes/charts v31.5.0, application image 26.5.0 (also reproduced on 26.2.1).

  • Redis backend: Dragonfly (Redis-compatible), not stock Redis
  • Connection: externalRedis in Helm values — redis://dragonfly:6379/1 with password
  • Consumer: sentry run consumer process-spans (processSpans deployment)

Related closed issue: #80440 (Dragonfly Lua randomseed — different failure mode).

Steps to Reproduce

  1. Deploy Sentry with externalRedis pointing at Dragonfly.
  2. Send performance/span data so the ingest-spans Kafka topic receives messages (or let backlog exist from SDK traffic).
  3. Observe process-spans consumer pod logs.

Expected Result

process-spans runs continuously, commits Kafka offsets, and performance traces appear in the UI.

Actual Result

Consumer crashes on every batch that hits span buffer metrics handling:

File "/usr/src/sentry/src/sentry/spans/buffer.py", line 473, in process_spans
    if evalsha_latency_ms > longest_evalsha_data[0]:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: '>' not supported between instances of 'bytes' and 'float'
  • Pod enters CrashLoopBackOff; ingest-spans offset does not advance.
  • Errors/issues pipeline is unaffected (separate consumers).
  • Stock Redis: we have not verified on Redis 7 yet; Dragonfly returns Lua numeric fields as bytes while Sentry expects a numeric type for evalsha_latency_ms.

Product Area

Performance Monitoring / ingest (process-spans consumer, spans buffer)

Version

26.5.0 (26.2.1 identical failure at buffer.py line 354)

Suggested fix

Coerce evalsha_latency_ms (and similar EVALSHA return fields) to float when isinstance(value, bytes) before comparison, or document Dragonfly as unsupported for the spans buffer until Redis semantics match.

Metadata

Metadata

Assignees

No one assigned
    No fields configured for issues without a type.

    Projects

    Status

    Waiting for: Product Owner

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions