ref(snuba): Port query subscriptions consumer to taskbroker raw mode#116288
Conversation
github-actions
Bot
added
the
Scope: Backend
untitaker
force-pushed
the
markusunterwaditzer/stream-1021-port-metrics-subscriptions-consumer-to-taskbroker
branch
from
fc6c689 to
8dab57b
Compare
| from sentry.snuba.query_subscriptions.consumer import handle_message | ||
| from sentry.utils import metrics |
There was a problem hiding this comment.
oh i misinterpreted this. you're asking why the import is in the fn, not claiming there is a circular import. the honest answer is that this is vibecoded. i'll move it out in a followup.
| _process_subscription_message(message_bytes, _d) | ||
|
|
||
|
|
||
| _register_subscription_tasks() |
There was a problem hiding this comment.
Why dynamically construct them when you're immediately calling the function? Can these tasks be statically defined?
There was a problem hiding this comment.
this dynamic code mainly enforces that PRs adding new datasets also create the respective task for it.
|
fun fact: entity is part of the message. so technically, we only need one task here, and the task could dispatch based on entity. we can still spawn many taskbrokers that spawn this task. the main reason we don't want this is because then all of our observability would break. our task timings would be muddled all together. the other reason is that this creates a weird relationship between tasks and pools. many tasks of the same type would now run on separate pools, which isn't something we do today. |
untitaker
deleted the
markusunterwaditzer/stream-1021-port-metrics-subscriptions-consumer-to-taskbroker
branch
2 participants
Adds taskbroker raw mode tasks for query subscription consumers (events, transactions, metrics, generic_metrics, eap).
ref STREAM-1021