Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

changefeedccl: support azure-event-hub:// for azure kafka streaming #115806

Merged
merged 3 commits into from
Mar 2, 2024
Merged

changefeedccl: support azure-event-hub:// for azure kafka streaming #115806

merged 3 commits into from
Mar 2, 2024

Commits on Mar 1, 2024

  1. changefeedccl: support azure-event-hub:// for azure kafka streaming 

    Previously, users had to navigate the complexities of obtaining azure event
hub's kafka endpoint and the corresponding sasl_user and sasl_password for azure
streaming.

This patch improves it to support a new scheme azure-event-hub:// with the
following syntax:

```
azure-event-hub://<NamespaceName>.servicebus.windows.net?shared_access_key_name=<KeyName>&shared_access_key=<KeyValue>
```

azure-event-hub:// can now be used to connect to kafka hosted on Azure event
hubs. The sinkURL must include mandatory parameters shared_access_key_name and
shared_access_key.   By default and by requirements, the options
"tls_enabled=true," "sasl_handshake=true," "sasl_enabled=true," and
"sasl_mechanism=PLAIN" are applied, as they are the only supported options.
Other parameters such as "topic_name", and "topic_prefix" are also supported.

Resolves: cockroachdb#103901, cockroachdb#110558

Release note (enterprise change): Changefeeds now support a new scheme
azure-event-hub:// for kafka data streaming to azure event hubs. The sinkURL
must include mandatory parameters shared_access_key_name and shared_access_key.
By default and by requirements, the options "tls_enabled=true,"
"sasl_handshake=true," "sasl_enabled=true," and "sasl_mechanism=PLAIN" are
applied, as they are the only supported options. Other parameters such as
"topic_name", and "topic_prefix" are also supported.

An example URI is:
```
azure-event-hub://myeventhubs.servicebus.windows.net:9093?shared_access_key_name=abc&shared_access_key=123
```
    @wenyihu6
    wenyihu6 committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    7f726b0 View commit details
    Browse the repository at this point in the history

  2. changefeedccl: redact shared_access_key from SHOW JOBS 

    Now that we support azure-event-hub:// scheme, this patch redacts
shared_access_key info for azure event hub from SHOW JOBS output.
See also: cockroachdb#103901
Epic: None
Release note: None
    @wenyihu6
    wenyihu6 committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    811ecd6 View commit details
    Browse the repository at this point in the history

  3. roachtest/cdc: add roachtest for azure-event-hub:// 

    This patch changes roachtest kafka-azure for azure event hub using the new
scheme azure-event-hub://.
    @wenyihu6
    wenyihu6 committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    7555372 View commit details
    Browse the repository at this point in the history