fix(kafka-headers): Make transaction forwarder header relevant #29618
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Now that the transaction forwarder is running in production and its offsets have caught up with current forwarder, its time to make the switch
fpacifici
approved these changes
Oct 28, 2021
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Context
We started sending the kafka header called
transaction_forwarderwith the value set to False in this #28908. The motivation was to start having the headers from the producer side so that we can deploy the transactions post process forwarder come online and catchup with the kafka offsets. The transactions post process forwarder was deployed using https://github.com/getsentry/ops/pull/4048. The kafka offsets have caught up. It's time to make the header value relevant so that errors forwarder only processes error messages and transactions forwarder starts processing transaction messages.Test
The offsets being processed by the 2 forwarders are mutually exclusive.
Errors post process forwarder logs with offsets being processed
19:01:49 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16000 offset 19:01:49 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16001 offset 19:01:49 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16003 offset 19:01:49 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16004 offset 19:01:49 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16007 offset 19:01:49 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16008 offset 19:01:49 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16009 offset 19:01:49 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16010 offset 19:01:49 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16011 offset 19:01:49 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16012 offset 19:01:49 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16014 offset 19:01:49 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16017 offset 19:01:49 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16018 offset 19:01:49 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16019 offset 19:01:49 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16021 offset 19:01:49 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16023 offset 19:01:49 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16024 offset 19:01:49 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16026 offset 19:01:49 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16027 offset 19:01:49 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16030 offsetTransactions post process forwarder logs with offsets being processed
19:01:48 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16002 offset 19:01:48 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16005 offset 19:01:48 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16006 offset 19:01:48 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16013 offset 19:01:48 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16015 offset 19:01:48 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16016 offset 19:01:48 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16020 offset 19:01:48 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16022 offset 19:01:48 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16025 offset 19:01:51 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16028 offset 19:01:51 [INFO] sentry.eventstream.kafka.postprocessworker: Process message with 16029 offset