ref(logging): configure grpc logger so its errors aren't dropped#116806
Merged
Conversation
Move the ingest-profiles taskbroker to the new kafka_clusters/kafka_topics format. Raw mode moves onto the profiles topic's per-topic raw block, and a retry topic is now mandatory for raw topics (raw messages aren't activations, so retries can't loop back into the raw topic). Retries are routed to the main taskworker topic so the default taskbroker picks them up instead of requiring another broker. Depends on taskbroker #668 (per-topic raw + mandatory-retry validation). ref STREAM-1093 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
sentry's LOGGING uses disable_existing_loggers=True and has no `grpc` logger entry. That makes grpc's own logging nondeterministic: if grpc's loggers are instantiated before this config is applied they get disabled (silent), and otherwise they propagate to root (console + a Sentry event). This matters for the push-mode taskworker, which runs a grpc server via taskbroker_client. When a PushTask body fails to deserialize, grpc logs "Exception deserializing request!" at ERROR on the `grpc._common` logger and aborts the call with INTERNAL — but whether that log is visible depends on import order. Configure `grpc` explicitly (ERROR, console, no propagation) so these failures land on the console deterministically, matching how other third-party loggers (arroyo, taskbroker_client, urllib3) are handled. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
github-actions
Bot
added
the
Scope: Backend
joshuarli
approved these changes
Jun 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
sentry's
LOGGINGusesdisable_existing_loggers: Trueand has nogrpclogger entry, so grpc's own logging is dropped or nondeterministically routed depending on import order. grpc catches a lot of exceptions internally (e.g. request deserialization failures in the push-mode taskworker) and logs them on thegrpc._commonlogger, so we want those to be reliably visible.Configure
grpcexplicitly —ERROR,console,propagate: False— matching how other third-party loggers (arroyo,taskbroker_client,urllib3) are handled.Context: getsentry/taskbroker#672 (closed) — originally attempted to log these in the client before realizing the fix belongs here.
🤖 Generated with Claude Code