KAFKA-15568: Use IncrementalAlterConfigs API to update the dynamic config of broker in ConfigCommand tool

[singhnama]

What
Use IncrementalAlterConfigs API to update the dynamic config of the broker in the ConfigCommand tool.

Why
As part of this KIP incrementalAlterConfigs API was introduced to change any config dynamically since the existing alterConfig API replaces any existing configuration with new configuration. This makes AlterConfigs unusable in cases where the client does not know the full existing configuration before making the modification. Even worse, in some cases, the client may be unable to discover the existing configuration. "Sensitive" fields are not returned by DescribeConfigs. kakfa-config.sh (CommandConfig) still uses alterConfig to update the config.

Testing
Updated existing test that will cover the cases.

[chia7712]

just curious. Are we going to remove the support from older kafka server? The PR is great but users will be unable to use the tool (3.8.0) to operate the older kafak (2.3.0-).

I assume the migration from alterConfigs to incrementalAlterConfigs should be addressed in 4.0.0. Please feel free to correct me :)

[singhnama]

just curious. Are we going to remove the support from older kafka server? The PR is great but users will be unable to use the tool (3.8.0) to operate the older kafak (2.3.0-).

thanks @chia7712 for the comment, Just for clarity, are we saying this because incrementalAlterConfigs was introduced in (2.3.0)? Would it be the case that someone running with older Kafka (2.3.0-) and using the updated tool?

[chia7712]

are we saying this because incrementalAlterConfigs was introduced in (2.3.0)?

yep

Would it be the case that someone running with older Kafka (2.3.0-) and using the updated tool?

yes, the "client", "server", and "tools" run with different versions in our env. However, our use cases are not the reason of blocking this PR. I'm just not sure whether the compatibility rules in community allows this changes.

@ijuma WDYT?

[dajac]

Thanks for the PR. There is actually a KIP for this change: KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh. Are you aware of it? cc @dengziming

[ijuma]

As David said, we decided it was best to go the KIP route. And because 4.0 is around the corner, it's a good release vehicle to make the incompatible change.

[singhnama]

thanks @dajac @ijuma @chia7712. I go through KIP-1011 and it talks about making the change compatible. and we already have pr #15304 open for that @dengziming working on it. So closing this.