Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fetcher: Skip transactional control batches #72

Merged
merged 2 commits into from
Aug 14, 2023

Conversation

JayH5
Copy link

@JayH5 JayH5 commented Aug 14, 2023

Back-port support for transactional control messages in kafka-python. Although longer-term we want to switch all consumers to confluent-kaka, this quick fix means that kafka-python-based consumers at least shouldn't crash if there are ever transactional producers writing to a consumed topic.

Note that this does not mean that the consumer supports transactions--there is still no support for the read_committed isolation.level (more here).

This code comes from dpkp#2361, original author bradenneal1. I didn't have a better way to write this myself. Unfortunately, writing a test for this would be quite difficult with the current test infrastructure. We have a history of back-porting stuff like this without tests, though (e.g. #71).

@JayH5
Copy link
Author

JayH5 commented Aug 14, 2023

We're a bit short of Yelp-org members today. I've received a +1 from abareghi [at] yelp [dot] com

@JayH5 JayH5 merged commit f6e8219 into master Aug 14, 2023
@JayH5 JayH5 deleted the u/jhewland/KAFKA-36342-skip-control-batches branch August 14, 2023 15:00
@gkousouris gkousouris mentioned this pull request Aug 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant