Skip to content

MINOR: Cherry-pick KAFKA-19851: Delete dynamic configs that were removed by Kafka to 4.1#22158

Merged
jsancio merged 2 commits intoapache:4.1from
tzy-0x7cf:4.1
May 7, 2026
Merged

MINOR: Cherry-pick KAFKA-19851: Delete dynamic configs that were removed by Kafka to 4.1#22158
jsancio merged 2 commits intoapache:4.1from
tzy-0x7cf:4.1

Conversation

@tzy-0x7cf
Copy link
Copy Markdown

@tzy-0x7cf tzy-0x7cf commented Apr 27, 2026

KAFKA-19851; Delete dynamic configs that were removed by Kafka (#21053)

When upgrading from Kafka 3.x to 4.0, the metadata log may contain
dynamic configurations that were removed in 4.0 (e.g.,
message.format.version per KIP-724). These removed configs cause
InvalidConfigurationException when users attempt to modify any
configuration, because validation checks all existing configs including
the removed ones.

Adds filtering to prevent unsupported or invalid configurations from
being applied during metadata replay. The filtering is implemented using
a SupportedConfigChecker interface that is injected via dependency
injection through Builder patterns. When a ConfigRecord is replayed, the
checker validates whether the configuration name is supported for the
given resource type. Unsupported configurations are silently ignored
during replay, ensuring that only valid configurations enter the
in-memory state.

The SupportedConfigChecker interface provides a default TRUE
implementation that accepts all configurations. The actual filtering
logic is implemented by DefaultSupportedConfigChecker, which maintains a
whitelist of valid configuration names per resource type (TOPIC,
CLIENT_METRICS, GROUP) based on the actual config definitions. The
filtering occurs in both ConfigurationDelta#replay and
ConfigurationControlManager#replay methods.

Added unit tests to ensure:

  • Removed configurations are filtered during the replay operations
  • Only supported configurations appear in the resulting metadata images
  • The filtering works correctly for all resource types (TOPIC, BROKER,
    CLIENT_METRICS, GROUP)
  • DefaultSupportedConfigChecker correctly identifies supported vs
    unsupported configurations for each resource type

Reviewers: José Armando García Sancio jsancio@apache.org, Jun Rao
junrao@apache.org, Alyssa Huang ahuang@confluent.io, Kevin Wu
kevin.wu2412@gmail.com, Andrew Grant agrant@confluent.io

(cherry picked from commit a35d649)
(cherry picked from commit 3be19e4)

0xffff-zhiyan and others added 2 commits April 27, 2026 10:27
…e#21053)

When upgrading from Kafka 3.x to 4.0, the metadata log may contain
dynamic configurations that were removed in 4.0 (e.g.,
message.format.version per KIP-724). These removed configs cause
InvalidConfigurationException when users attempt to modify any
configuration, because validation checks all existing configs including
the removed ones.

Adds filtering to prevent unsupported or invalid configurations from
being applied during metadata replay. The filtering is implemented using
a SupportedConfigChecker interface that is injected via dependency
injection through Builder patterns. When a ConfigRecord is replayed, the
checker validates whether the configuration name is supported for the
given resource type. Unsupported configurations are silently ignored
during replay, ensuring that only valid configurations enter the
in-memory state.

The SupportedConfigChecker interface provides a default TRUE
implementation that accepts all configurations. The actual filtering
logic is implemented by DefaultSupportedConfigChecker, which maintains a
whitelist of valid configuration names per resource type (TOPIC,
CLIENT_METRICS, GROUP) based on the actual config definitions. The
filtering occurs in both ConfigurationDelta#replay and
ConfigurationControlManager#replay methods.

Added unit tests to ensure:
- Removed configurations are filtered during the replay operations
- Only supported configurations appear in the resulting metadata images
- The filtering works correctly for all resource types (TOPIC, BROKER,
CLIENT_METRICS, GROUP)
- DefaultSupportedConfigChecker correctly identifies supported vs
unsupported configurations for each resource type

Reviewers: José Armando García Sancio <jsancio@apache.org>, Jun Rao
<junrao@apache.org>, Alyssa Huang <ahuang@confluent.io>, Kevin Wu
<kevin.wu2412@gmail.com>, Andrew Grant <agrant@confluent.io>

(cherry picked from commit a35d649)
(cherry picked from commit 3be19e4)
@jsancio jsancio merged commit 639e147 into apache:4.1 May 7, 2026
29 of 31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants