Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

changefeedccl: allow per changefeed kafka quota config #118643

Merged
merged 1 commit into from
Feb 9, 2024

Commits on Feb 9, 2024

  1. changefeedccl: allow per changefeed kafka quota config

    Previously, users were limited to setting a single kafka quota configuration for
    cockroachdb which was then applied and restricting all changefeeds. This patch
    introduces a new changefeed configuration option, allowing users to define
    client id for different changefeeds, allowing users to specify different kafka
    quota configurations for different changefeeds. To use it, users can specify a
    unique client ID using `kafka_sink_config` and configure different quota
    settings on kafka server based on
    https://kafka.apache.org/documentation/#quotas.
    
    ```
    CREATE CHANGEFEED FOR foo WITH kafka_sink_config='{"ClientID": "clientID1"}'
    ```
    
    Note that Fixes: cockroachdb#92290
    
    Release note: `kafka_sink_config` now supports specifying a different client ID
    for different changefeeds, enabling users to define distinct kafka quota
    configurations for various changefeeds.
    
    For any kafka versions >= V1_0_0_0 ([KIP-190: Handle client-ids consistently
    between clients and
    brokers](https://cwiki.apache.org/confluence/display/KAFKA/KIP-190%3A+Handle+client-ids+consistently+between+clients+and+brokers)),
    any string can be used as client ID. For earlier kafka versions, clientID can
    only contain characters [A-Za-z0-9._-] are acceptable.
    
    For example,
    ```
    CREATE CHANGEFEED FOR ... WITH kafka_sink_config='{"ClientID": "clientID1"}'
    ```
    wenyihu6 committed Feb 9, 2024
    Configuration menu
    Copy the full SHA
    9a05851 View commit details
    Browse the repository at this point in the history