fix(taskworker): Improve Queue Size Metrics

[george-sentry]

Linear

Completes STREAM-910

Description

• Right now, the task queue size is only emitted on calls to the PushTask RPC endpoint, which means if a worker isn't being used, it won't be emitting its queue size metrics, giving us no insight into idle workers
• We aren't emitting metrics for the result queue size, which is also very important
• Finally, we aren't tagging queue size metrics by pod name, so the most we can do right now is an average across an entire processing pool, which is only marginally helpful

This PR fixes those things 💪

[sentry]

Bug: The TaskWorker class does not accept a pod_name parameter, causing queue size metrics to be incorrectly tagged with pod_name="unknown".
_{Severity: MEDIUM}

Suggested Fix

Update the TaskWorker.__init__ method to accept a pod_name parameter and pass it to the TaskWorkerProcessingPool constructor, similar to how PushTaskWorker handles it.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: clients/python/src/taskbroker_client/worker/worker.py#L130-L133

Potential issue: The `TaskWorker` class, used for pull-mode workers, does not accept a
`pod_name` parameter in its `__init__` method. Consequently, it cannot pass the pod name
when it instantiates `TaskWorkerProcessingPool`. The `TaskWorkerProcessingPool` then
defaults the `pod_name` to `"unknown"`. This results in all queue size metrics for
pull-mode workers being tagged with `pod_name="unknown"`, making it difficult to monitor
queue sizes on a per-pod basis for this worker type.

_{Did we get this right? 👍 / 👎 to inform future reviews.}

[george-sentry]

I don't care about pull workers.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 8d8f444. Configure here.}

[cursor]

TaskWorker missing pod_name parameter for metrics tagging

Medium Severity

The new pod_name parameter was added to PushTaskWorker and plumbed through to TaskWorkerProcessingPool, but the TaskWorker class (PULL mode) was not updated. Since TaskWorkerProcessingPool.result_thread now emits gauge metrics tagged with pod_name, all PULL-mode workers will always report pod_name="unknown", defeating the PR's goal of per-pod metric visibility for those workers.

Additional Locations (1)

• clients/python/src/taskbroker_client/worker/worker.py#L129-L156

 

^{Reviewed by Cursor Bugbot for commit 8d8f444. Configure here.}

[george-sentry]

I don't care about pull workers.

[linear-code]

STREAM-910 Improve Worker Queue Size Metrics

[evanh]

Approved as is but would be better if pod name can be automatically added.

[george-sentry]

Approved as is but would be better if pod name can be automatically added.

Agreed, will look into this in the future.