fix(sentry): factor out redis-related logging#563
Conversation
As I have enabled log-collection to Sentry, I have noticed that the logs are massively spammed by the queue-processing logging, e.g., • “Waiting for tasks from rebase_queue_c10s (timeout: 30s)...” • “No tasks received, continuing to wait...” Therefore I introduce a separate logger for these highly recurrent messages that can be explicitly ignored by Sentry. Signed-off-by: Matej Focko <mfocko@packit.dev>
There was a problem hiding this comment.
Code Review
This pull request introduces a dedicated redis_logger ('agent.redis') across multiple agents and configures Sentry to ignore it, aiming to reduce log noise from repetitive Redis polling. The reviewer recommends keeping the connection and task receipt log messages on the main logger rather than moving them to redis_logger. Since these events are not spammy and occur only at startup or when a task is actually dispatched, retaining them on the main logger preserves valuable Sentry breadcrumbs for debugging, while still successfully filtering out the idle polling logs.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| redis_logger.info( | ||
| f"Connected to Redis, max retries set to {max_retries}, listening to queue: {backport_queue}" | ||
| ) | ||
|
|
||
| while True: | ||
| logger.info(f"Waiting for tasks from {backport_queue} (timeout: 30s)...") | ||
| redis_logger.info(f"Waiting for tasks from {backport_queue} (timeout: 30s)...") | ||
| element = await fix_await(redis.brpop([backport_queue], timeout=30)) | ||
| if element is None: | ||
| logger.info("No tasks received, continuing to wait...") | ||
| redis_logger.info("No tasks received, continuing to wait...") | ||
| continue | ||
|
|
||
| _, payload = element | ||
| logger.info("Received task from queue.") | ||
| redis_logger.info("Received task from queue.") |
There was a problem hiding this comment.
The Connected to Redis... and Received task from queue. log messages are not highly recurrent or spammy. The connection log occurs only once at startup, and the task receipt log occurs only when a task is actually dispatched. Keeping these on the main logger is highly beneficial for observability and debugging in Sentry (e.g., as breadcrumbs leading up to an error), whereas only the idle polling logs (Waiting for tasks... and No tasks received...) should be ignored.
| redis_logger.info( | |
| f"Connected to Redis, max retries set to {max_retries}, listening to queue: {backport_queue}" | |
| ) | |
| while True: | |
| logger.info(f"Waiting for tasks from {backport_queue} (timeout: 30s)...") | |
| redis_logger.info(f"Waiting for tasks from {backport_queue} (timeout: 30s)...") | |
| element = await fix_await(redis.brpop([backport_queue], timeout=30)) | |
| if element is None: | |
| logger.info("No tasks received, continuing to wait...") | |
| redis_logger.info("No tasks received, continuing to wait...") | |
| continue | |
| _, payload = element | |
| logger.info("Received task from queue.") | |
| redis_logger.info("Received task from queue.") | |
| logger.info( | |
| f"Connected to Redis, max retries set to {max_retries}, listening to queue: {backport_queue}" | |
| ) | |
| while True: | |
| redis_logger.info(f"Waiting for tasks from {backport_queue} (timeout: 30s)...") | |
| element = await fix_await(redis.brpop([backport_queue], timeout=30)) | |
| if element is None: | |
| redis_logger.info("No tasks received, continuing to wait...") | |
| continue | |
| _, payload = element | |
| logger.info("Received task from queue.") |
| redis_logger.info( | ||
| f"Connected to Redis, max retries set to {max_retries}, listening to queue: {rebase_queue}" | ||
| ) | ||
|
|
||
| while True: | ||
| logger.info(f"Waiting for tasks from {rebase_queue} (timeout: 30s)...") | ||
| redis_logger.info(f"Waiting for tasks from {rebase_queue} (timeout: 30s)...") | ||
| element = await fix_await(redis.brpop([rebase_queue], timeout=30)) | ||
| if element is None: | ||
| logger.info("No tasks received, continuing to wait...") | ||
| redis_logger.info("No tasks received, continuing to wait...") | ||
| continue | ||
|
|
||
| _, payload = element | ||
| logger.info("Received task from queue.") | ||
| redis_logger.info("Received task from queue.") |
There was a problem hiding this comment.
The Connected to Redis... and Received task from queue. log messages are not highly recurrent or spammy. The connection log occurs only once at startup, and the task receipt log occurs only when a task is actually dispatched. Keeping these on the main logger is highly beneficial for observability and debugging in Sentry (e.g., as breadcrumbs leading up to an error), whereas only the idle polling logs (Waiting for tasks... and No tasks received...) should be ignored.
| redis_logger.info( | |
| f"Connected to Redis, max retries set to {max_retries}, listening to queue: {rebase_queue}" | |
| ) | |
| while True: | |
| logger.info(f"Waiting for tasks from {rebase_queue} (timeout: 30s)...") | |
| redis_logger.info(f"Waiting for tasks from {rebase_queue} (timeout: 30s)...") | |
| element = await fix_await(redis.brpop([rebase_queue], timeout=30)) | |
| if element is None: | |
| logger.info("No tasks received, continuing to wait...") | |
| redis_logger.info("No tasks received, continuing to wait...") | |
| continue | |
| _, payload = element | |
| logger.info("Received task from queue.") | |
| redis_logger.info("Received task from queue.") | |
| logger.info( | |
| f"Connected to Redis, max retries set to {max_retries}, listening to queue: {rebase_queue}" | |
| ) | |
| while True: | |
| redis_logger.info(f"Waiting for tasks from {rebase_queue} (timeout: 30s)...") | |
| element = await fix_await(redis.brpop([rebase_queue], timeout=30)) | |
| if element is None: | |
| redis_logger.info("No tasks received, continuing to wait...") | |
| continue | |
| _, payload = element | |
| logger.info("Received task from queue.") |
| redis_logger.info( | ||
| f"Connected to Redis, max retries set to {max_retries}, listening to queue: {rebuild_queue}" | ||
| ) | ||
|
|
||
| while True: | ||
| logger.info(f"Waiting for tasks from {rebuild_queue} (timeout: 30s)...") | ||
| redis_logger.info(f"Waiting for tasks from {rebuild_queue} (timeout: 30s)...") | ||
| element = await fix_await(redis.brpop([rebuild_queue], timeout=30)) | ||
| if element is None: | ||
| logger.info("No tasks received, continuing to wait...") | ||
| redis_logger.info("No tasks received, continuing to wait...") | ||
| continue | ||
|
|
||
| _, payload = element | ||
| logger.info("Received task from queue.") | ||
| redis_logger.info("Received task from queue.") |
There was a problem hiding this comment.
The Connected to Redis... and Received task from queue. log messages are not highly recurrent or spammy. The connection log occurs only once at startup, and the task receipt log occurs only when a task is actually dispatched. Keeping these on the main logger is highly beneficial for observability and debugging in Sentry (e.g., as breadcrumbs leading up to an error), whereas only the idle polling logs (Waiting for tasks... and No tasks received...) should be ignored.
| redis_logger.info( | |
| f"Connected to Redis, max retries set to {max_retries}, listening to queue: {rebuild_queue}" | |
| ) | |
| while True: | |
| logger.info(f"Waiting for tasks from {rebuild_queue} (timeout: 30s)...") | |
| redis_logger.info(f"Waiting for tasks from {rebuild_queue} (timeout: 30s)...") | |
| element = await fix_await(redis.brpop([rebuild_queue], timeout=30)) | |
| if element is None: | |
| logger.info("No tasks received, continuing to wait...") | |
| redis_logger.info("No tasks received, continuing to wait...") | |
| continue | |
| _, payload = element | |
| logger.info("Received task from queue.") | |
| redis_logger.info("Received task from queue.") | |
| logger.info( | |
| f"Connected to Redis, max retries set to {max_retries}, listening to queue: {rebuild_queue}" | |
| ) | |
| while True: | |
| redis_logger.info(f"Waiting for tasks from {rebuild_queue} (timeout: 30s)...") | |
| element = await fix_await(redis.brpop([rebuild_queue], timeout=30)) | |
| if element is None: | |
| redis_logger.info("No tasks received, continuing to wait...") | |
| continue | |
| _, payload = element | |
| logger.info("Received task from queue.") |
| redis_logger.info(f"Connected to Redis, max retries set to {max_retries}") | ||
|
|
||
| while True: | ||
| logger.info("Waiting for tasks from triage_queue (timeout: 30s)...") | ||
| redis_logger.info("Waiting for tasks from triage_queue (timeout: 30s)...") | ||
| element = await fix_await(redis.brpop([RedisQueues.TRIAGE_QUEUE.value], timeout=30)) | ||
| if element is None: | ||
| logger.info("No tasks received, continuing to wait...") | ||
| redis_logger.info("No tasks received, continuing to wait...") | ||
| continue | ||
|
|
||
| _, payload = element | ||
| logger.info("Received task from queue") | ||
| redis_logger.info("Received task from queue") |
There was a problem hiding this comment.
The Connected to Redis... and Received task from queue log messages are not highly recurrent or spammy. The connection log occurs only once at startup, and the task receipt log occurs only when a task is actually dispatched. Keeping these on the main logger is highly beneficial for observability and debugging in Sentry (e.g., as breadcrumbs leading up to an error), whereas only the idle polling logs (Waiting for tasks... and No tasks received...) should be ignored.
| redis_logger.info(f"Connected to Redis, max retries set to {max_retries}") | |
| while True: | |
| logger.info("Waiting for tasks from triage_queue (timeout: 30s)...") | |
| redis_logger.info("Waiting for tasks from triage_queue (timeout: 30s)...") | |
| element = await fix_await(redis.brpop([RedisQueues.TRIAGE_QUEUE.value], timeout=30)) | |
| if element is None: | |
| logger.info("No tasks received, continuing to wait...") | |
| redis_logger.info("No tasks received, continuing to wait...") | |
| continue | |
| _, payload = element | |
| logger.info("Received task from queue") | |
| redis_logger.info("Received task from queue") | |
| logger.info(f"Connected to Redis, max retries set to {max_retries}") | |
| while True: | |
| redis_logger.info("Waiting for tasks from triage_queue (timeout: 30s)...") | |
| element = await fix_await(redis.brpop([RedisQueues.TRIAGE_QUEUE.value], timeout=30)) | |
| if element is None: | |
| redis_logger.info("No tasks received, continuing to wait...") | |
| continue | |
| _, payload = element | |
| logger.info("Received task from queue") |
|
As for the Gemini review:
|
actually, now that I think about it… Sentry allows also collection of metrics, so maybe it would be better to track the proportions of the tasks (received, retried, success, failed), instead of pushing the repeating log message to logs? |
As I have enabled log-collection to Sentry, I have noticed that the logs are massively spammed by the queue-processing logging, e.g.,
• “Waiting for tasks from rebase_queue_c10s (timeout: 30s)...”
• “No tasks received, continuing to wait...”
Therefore I introduce a separate logger for these highly recurrent messages that can be explicitly ignored by Sentry.
TODO
logging.getLogger(__name__)is not very helpful, since in all cases it gets expanded to just__main__, I think it would be for the better to adjust that