Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KIP-339: Add Incremental Config updates API #1966

Merged
merged 2 commits into from
Jun 14, 2021

Conversation

ajanikow
Copy link
Contributor

@ajanikow ajanikow commented Jun 14, 2021

Hello!

In this PR I cover IncrementalAlterConfigs API in Sarama. It is very useful, due to fact we can specify which config entry needs to be affected (instead of sending everything over and over again).

Fixes: #1830, birdayz/kaf#155

Kafka Code Links:

Best Regards,
Adam.

@ajanikow ajanikow requested a review from bai as a code owner June 14, 2021 08:57
@dnwe dnwe changed the title Add Incremental Config updates API KIP-339: Add Incremental Config updates API Jun 14, 2021
@ajanikow
Copy link
Contributor Author

Hello @dnwe!

Fixing tests - response in case of AlterConfigs and IncrementalAlterConfigs is the same, thats why tests passed.

Best Regards,
Adam.

// IncrementalAlterConfigsResponse is a response type for incremental alter config
type IncrementalAlterConfigsResponse struct {
ThrottleTime time.Duration
Resources []*AlterConfigsResourceResponse
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this place I have used AlterConfigsResourceResponse - thats why file with according definition is also affected. In same way it is used in Kafka Java Client.

Copy link
Collaborator

@dnwe dnwe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ajanikow welcome and thanks for another great PR ⭐

I've reviewed the changes and I'm happy that the protocol matches with the V0 types for request and response as defined in the protocol spec. Thanks for adding test cases to cover the roundtripping of the request and response types.

Lets go ahead and merge this PR now so that people can start consuming it. If you have some time, there's a couple of interesting follow-up PRs that would be great if you could also take on.

It would be good to add something to admin.go to expose the per-broker IncrementalAlterConfigs func call in the same way as we currently expose the regular one via AlterConfig in admin.go
https://github.com/Shopify/sarama/blob/d01b3443aa926c7a04f00182002e77fea420ac57/admin.go#L666-L716

I also wondered if you might be able to add a functional test (see the various functional_ prefixed files) that drives this API against a real Kafka cluster so we can be confident that the functionality is working and doesn't regress in the future

@ajanikow
Copy link
Contributor Author

Hello!

For integration tests it is not a problem, already have them in my project. This is very useful in case of Broker Dynamic Configuration (can be tested to, for example, add listener in runtime)

Will add them together with admin.go modifications.

Best Regards,
Adam.

@dnwe dnwe merged commit a5128bc into IBM:master Jun 14, 2021
@fengyinqiao
Copy link
Contributor

Hello!

For integration tests it is not a problem, already have them in my project. This is very useful in case of Broker Dynamic Configuration (can be tested to, for example, add listener in runtime)

Will add them together with admin.go modifications.

Best Regards, Adam.

Will add them together with admin.go modifications. Have you done this? Looking forward to your reply~

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

How to remove Kafka Topic Config entry using API? (OpType.DELETE from kafka-config cli)
3 participants