Skip to content

feat(devservices): Add containerized taskbroker to devservices config #92

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Dec 12, 2024
Merged

feat(devservices): Add containerized taskbroker to devservices config #92

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
92584b4
add containerized taskbroker
hubertdeng123 Dec 11, 2024
2885dfa
add network settings to taskbroker
hubertdeng123 Dec 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion devservices/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,23 @@ x-sentry-service-config:
repo_name: sentry-shared-kafka
branch: main
repo_link: https://github.com/getsentry/sentry-shared-kafka
taskbroker:
description: Taskbroker service
modes:
default: [kafka]
containerized: [kafka]
containerized: [kafka, taskbroker]

services:
taskbroker:
image: ghcr.io/getsentry/taskbroker:latest
environment:
TASKBROKER_KAFKA_CLUSTER: "kafka-kafka-1:9093"
Copy link
Member

@evanh evanh Dec 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this address get set correctly? I notice it's different from the one defined in snuba, and different from the name of the current kafka devservice (which is called sentry_kafka).

Copy link
Member

@markstory markstory Dec 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the new devservices command uses different host names. I've also noticed that the devservices container has 9092 exposed instead of 9093.

Copy link
Member Author

@hubertdeng123 hubertdeng123 Dec 12, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should actually be fine, we're using mostly the same settings as before and taskbroker should be communicating on the docker internal network which should use port 9093 as defined here: https://github.com/getsentry/sentry-shared-kafka/blob/c8ab4ee7627d435c4c5ebccf21c5744950eac1f6/devservices/config.yml#L25.

If taskbroker cannot properly connect to kafka, will it error out anywhere? How can we tell it is properly configured?

Copy link
Member

@markstory markstory Dec 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, taskbroker will error and crash if it cannot connect to Kafka.

ports:
- 127.0.0.1:50051:50051
networks:
- devservices
restart: unless-stopped
platform: linux/amd64

networks:
devservices:
Expand Down
Loading