Skip to content

Forward telemetry in a separate thread#1973

Merged
alexmojaki merged 73 commits into
mainfrom
alex/forwarder-thread-base
May 26, 2026
Merged

Forward telemetry in a separate thread#1973
alexmojaki merged 73 commits into
mainfrom
alex/forwarder-thread-base

Conversation

@alexmojaki
Copy link
Copy Markdown
Collaborator

logfire.experimental.forwarding now forwards OTLP payloads asynchronously.

The forward_export_request() and logfire_proxy() helpers now enqueue accepted payloads locally and return immediately instead of waiting for Logfire. This prevents Logfire outages or slow responses from blocking application request handling.

Key user-visible changes:

  • Accepted requests return local OTLP success, not confirmation that Logfire received the payload.
  • Queue-full or post-shutdown requests return OTLP partial success.
  • Each Logfire config manages one queue per destination Logfire URL (usually only one).
  • The queues have hard limits of 1000 items and 64mb.
  • Forwarding now uses all configured Logfire write tokens, not just the first.
  • Only application/x-protobuf and application/json OTLP payloads are accepted, so that the response can match.
  • Only content-type, user-agent, and content-encoding from the original request are forwarded, whereas previously all headers were forwarded except for a specific blacklist.
  • body can no longer be None.
  • Uses the same disk-backed retrying mechanism as normal exports instead of only trying once.

Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found

Confidence score: 3/5

  • There is a concrete medium-risk behavior in logfire/_internal/config.py: the shutdown path may lazy-initialize the variable provider, which can trigger remote variable machinery during teardown instead of cleanly shutting down.
  • A secondary timing issue in logfire/_internal/config.py uses wall-clock time for timeout budgeting; clock changes could skew remaining time and cause premature or delayed timeout handling.
  • Given one high-confidence 6/10 issue affecting shutdown behavior, this carries some regression risk and should be addressed before relying on this as a low-risk merge.
  • Pay close attention to logfire/_internal/config.py - shutdown/provider invocation and timeout budget calculations are the key risk points.

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread logfire/_internal/config.py Outdated
Comment thread logfire/_internal/config.py Outdated
@alexmojaki alexmojaki merged commit a065b31 into main May 26, 2026
18 checks passed
@alexmojaki alexmojaki deleted the alex/forwarder-thread-base branch May 26, 2026 16:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant