Skip to content

[Feature](Kafka) Add kerberos support for kafka#14431

Merged
morningman merged 1 commit intoapache:masterfrom
amosbird:kafka_krb
Nov 21, 2022
Merged

[Feature](Kafka) Add kerberos support for kafka#14431
morningman merged 1 commit intoapache:masterfrom
amosbird:kafka_krb

Conversation

@amosbird
Copy link
Contributor

@amosbird amosbird commented Nov 20, 2022

Proposed changes

Issue Number: close #14434 #14353

Problem summary

Compile librdkafka with Kerberos SASL GSSAPI support. cc @xiaokang

Checklist(Required)

  1. Does it affect the original behavior:
    • Yes
    • No
    • I don't know
  2. Has unit tests been added:
    • Yes
    • No
    • No Need
  3. Has document been added or modified:
    • Yes
    • No
    • No Need
  4. Does it need to update dependencies:
    • Yes
    • No
  5. Are there any changes that cannot be rolled back:
    • Yes (If Yes, please explain WHY)
    • No

@amosbird amosbird changed the title Add kerberos support for kafka [Feature](Kafka) Add kerberos support for kafka Nov 20, 2022
@xiaokang
Copy link
Contributor

Thanks @amosbird for fixing the issuse!

A issue #14434 is created for others to search the problem. You can link to the issue.

CPPFLAGS="-I${TP_INCLUDE_DIR}" \
LDFLAGS="-L${TP_LIB_DIR}" \
./configure --prefix="${TP_INSTALL_DIR}" --enable-static --enable-shared=no --with-openssl="${TP_INSTALL_DIR}" --with-pic
./configure --prefix="${TP_INSTALL_DIR}" --enable-static --enable-shared=no --with-openssl="${TP_INSTALL_DIR}" --with-pic --enable-gssapi="${TP_INSTALL_DIR}" --with-gss_impl=mit --with-dblib=none
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is '--with-dblib=none' needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It helps exclude unneeded system dependencies (gdm in my case).

# NOTE(amos): librdkafka uses a weird autoconf variant (mklove) which doesn't allow extending PKG_CONFIG with spaces in cmd.
# As a result, we use a patch to hard code "--static" into PKG_CONFIG instead.
# PKG_CONFIG="pkg-config --static"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to add --enable-gssapi for build_librdkafka?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so. Linking to sasl2 will pull in all required archives.

@xiaokang
Copy link
Contributor

LGTM

@xiaokang
Copy link
Contributor

We have tested in an env with kafka kerberos authentication, the issue #14431 is fixed.

Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 21, 2022
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@morningman morningman merged commit 7624c80 into apache:master Nov 21, 2022
luwei16 pushed a commit to luwei16/Doris that referenced this pull request Apr 7, 2023
…e#1170)

Compile librdkafka with Kerberos SASL GSSAPI support.

Co-authored-by: Amos Bird <amosbird@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] kafka kerberos authentication error 'missing a GSSAPI module' in routine load

3 participants