Kerberos Authentication for Elasticsearch

[andrewkroh]

Elasticsearch (by way of X-Pack) is adding support for authenticating via Kerberos. As a Beats user I'd like to be able to use Kerberos for authenticating with the Elasticsearch output cluster (or monitoring cluster).

There is a pure Go client that is Apache 2.0 licensed -- https://github.com/jcmturner/gokrb5. We should test this to see if it meets our requirements. If not there are a few other libraries (mostly that require cgo).

Based on the ES ticket it sounds like the flow will be to authenticate to ES with the Kerberos ticket and exchange it for an Elasticsearch token. Then use the ES token when making the _bulk requests.

Related Tickets

• Elasticsearch - Kerberos support elasticsearch#30243
• Logstash - Logstash to support SPNEGO + Kerberos authentication to Elasticsearch  logstash#9785

[andrewkroh]

It does look like the https://github.com/jcmturner/gokrb5 library should support what we need to integrated with ES:

• authenticate with the keytab
• generate a SPNEGO ticket

For testing purposes we can setup dockerized kerberos environment. Here's a RedHat guide: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system-level_authentication_guide/configuring_a_kerberos_5_server

I've been told another good resource for getting started is https://github.com/freeipa/freeipa-workshop.

[andrewkroh]

The ES team has a test fixture for Kerberos at https://github.com/elastic/elasticsearch/tree/master/test/fixtures/krb5kdc-fixture.

In Kerberos auth feature in ES is being worked from a feature branch at: https://github.com/elastic/elasticsearch/commits/feature/kerberos

[andresrc]

Support provided in #17927

@kvch what's missing to close this issue? thanks!

[kvch]

The mering and backporting of the automated tests: #18127
Hopefully, it will be done today or tomorrow.

[kvch]

I've merged and backported the automated tests.

[ovictorain]

Hi @kvch I encounterd the same problem mentioned here: #IBM/sarama#1658, did you find a workaroud?
BTW, as a beats user, I don't think it's elegant to modify sarama source code and reimport 😌. thx
(I tested 43f5689 and problem still remains)

[kvch]

@ovictorain Thanks for reporting it. Do you mind opening a separate issue, so we can track it? The fix is not yet integrated into Beats.

I agree it is not elegant. My plan is to open a PR with the revert. But until it gets merged upstream, we have no choice but to use a fork of the repository. :) I rather not keep our users waiting. :)

[kvch]

@ovictorain Now I am looking at the issue you commented on. Kerberos authentication for Elasticserach has nothing to do with Kafka. What output are you using? What is the error you are getting?