chore(taskworker): Move devenv for profiles consumer to taskbroker#116194
Conversation
fix STREAM-1041 Remove the profiles consumer from sentry devserver (that cannot be supported, up to devinfra to make that happen with their own devservices stuff), and introduce a new taskbroker + worker instance to replace it. Taskbroker eventually will get a way to read from many topics, which would eliminate these extra containers entirely. Next steps are to roll this out to s4s2, then to prod, and finally remove the ingest-profiles consumer from code.
github-actions
Bot
added
the
Scope: Backend
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit dfbe45d. Configure here.
…ll modes The ingest-profiles passthrough taskbroker needs its corresponding taskworker to process messages, matching the profiling mode configuration.
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
| # via devservices (STREAM-1041) | ||
|
|
There was a problem hiding this comment.
Bug: Running sentry devserver --workers without the ingest-profiles devservice will cause profiling data to be silently dropped due to a missing dependency check.
Severity: MEDIUM
Suggested Fix
Add a validation check at devserver startup when --workers is used. The check should verify that the ingest-profiles devservice (or its underlying components like the taskbroker on port 50052) is running. If it's not, log a clear warning or raise an error to inform the developer that profiling data will not be processed and guide them on how to start the required service.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: src/sentry/runner/commands/devserver.py#L336-L337
Potential issue: The responsibility for processing profiling data has been moved from
the devserver to a separate `ingest-profiles` devservice. However, when running `sentry
devserver --workers`, the system does not validate whether this required service is
active. If a developer starts the devserver without the corresponding `ingest-profiles`
service, profiling data sent to the Kafka topic will be silently dropped because no
consumer is present to process it. This can lead to confusion and data loss during local
development, as there are no warnings or errors indicating the incomplete setup.
Did we get this right? 👍 / 👎 to inform future reviews.
2 participants
fix STREAM-1041
Remove the profiles consumer from sentry devserver (that cannot be
supported, up to devinfra to make that happen with their own devservices
stuff), and introduce a new taskbroker + worker instance to replace it.
Taskbroker eventually will get a way to read from many topics, which
would eliminate these extra containers entirely.
Next steps are to roll this out to s4s2, then to prod, and finally
remove the ingest-profiles consumer from code.