kafka-python zstd compression #2015
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gabriel-tincu
changed the title
gabriel-tincu
force-pushed
the
gabi-kafka-client-zstd-compression
branch
2 times, most recently
from
2b84da7
to
bb20e65
Compare
gabriel-tincu
force-pushed
the
gabi-kafka-client-zstd-compression
branch
6 times, most recently
from
7c9b4c3
to
c7be6c0
Compare
jeffwidman
requested changes
Mar 15, 2020
There was a problem hiding this comment.
Thanks for all your hard work here! Two 👍 on most of what I saw and looking forward to merging this.
It actually looks to me like this might be better broken into multiple PR's:
- Add 2.1.0 to testing matrix
- Add
log_start_offsetto theRecordMetadata(This could potentially be included with above, but I suspect it's possible to separate them and might be easier to review/discuss that way) - Add support for the additional
Producer[Request/Response]classes including wiring them into the producer. - Add
zstdsupport.
Again, most of this looks like it's done and the general code implementation looked solid to me, it's just easier to review/discuss/tweak the PR's if the scope of each one is a bit more focused rather than trying to do everything in one big PR.
Your thoughts?
Lastly, looks like there's a consistent test failure with the Kafka 0.10.2.2 versions... not sure why, that's partially why I suggest breaking it up as it will quickly become apparent which set of changes is breaking this...
gabriel-tincu
force-pushed
the
gabi-kafka-client-zstd-compression
branch
7 times, most recently
from
150c8b3
to
cc3c3fd
Compare
…ependency file, readme and codec checks add 2.1.0 to possible releases being tested / downloadable, skip zstd tests when version is too old add zstd tests to producer tests, add zstandard to tox dep list add New produce request/response classes to support up to V8 produce,(V7 is required for zstd compression) add ProduceResponse fields in use after V5 force broker version onto producers on ver 2.4.0 in unittests, due to inconsitencies with the env var for it produce response api V2 included in the same logic branch as 3 and 5 add proper docstrings to produce request / response classes impose limit on decompress output size on a failed decompress attemtp update producer tests to be more idiomatic with the testing framework pass merged error message ahead, keep old api
gabriel-tincu
force-pushed
the
gabi-kafka-client-zstd-compression
branch
from
cc3c3fd
to
d277280
Compare
gabriel-tincu
force-pushed
the
gabi-kafka-client-zstd-compression
branch
from
f628658
to
73197b2
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Update unit tests, add relevant functions in the codec
Fix #1791
This change is