SASL Authentication

[nadeevkm]

Greeting! Is it possible to use kafka-webview with a cluster with a SASL authenticaton? In simple console consumer I can do this by just passing jaas conf(with login/password) file as JVM argument like

export KAFKA_OPTS="-Djava.security.auth.login.config=/home/nkm/Apps/kafka_2.11-2.0.0/config/jaas_client.conf"

and consumer.property file with lines

security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN

as cmd arguments, like..

bin/kafka-console-consumer.sh --bootstrap-server localhost:9029092 --topic test_topic --from-beginning --consumer.config config/consumer.properties

[Crim]

Do you know of a good online resource/tutorial for setting up SASL authentication on brokers? If I can configure it locally I can likely work out how to include it in the app.

[thammuio]

@Crim - I can help with this. What distro of Kafka you need? (Apache/Horotonworks/Cloudera/Confluent)...

This is a good doc which covers all vendor distros... Probably you need to setup a KDC, if you don't have one already.

[Crim]

just standard Apache's Kafka, or whatever is easiest to get up and going really.

Thanks!

[nadeevkm]

Glad to hear, it would be great!

As far as SASL is concerned - it's quite a simple thing when you get it, so I can describe it here.

To setup SASL authentification in a kafka cluster you shoul fulfill several steps:

1. Add relevant properties in properies files
2. Make jaas-file with authentication information
3. Pass JVM a special environment parametr
4. Run zookeeper/kafka/producers/consumers explicitly passing modifyed config files

Now in details:

1. Zookeper

add this at config/zookeeper.properies

authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
requireClientAuthScheme=sasl
jaasLoginRenew=3600000

make jaas file, for example config/jaas_zookeeper.conf with lines

Server {
   org.apache.zookeeper.server.auth.DigestLoginModule required
   username="admin"
   password="admin-secret"
   user_zooclient="client-secret";
};

pass JVM a special param, in linux you can do it this way:

export SERVER_JVMFLAGS="-Djava.security.auth.login.config=/home/nkm/apps/kafka_2.11-2.0.0/config/jaas_zookeeper.conf"

and run zookeeper

bin/zookeeper-server-start.sh config/zookeeper.properties

2. Kafka broker

modify or add this properies at config/server.properies

advertised.host.name=localhost
advertised.listeners=SASL_PLAINTEXT://localhost:9092

allow.everyone.if.no.acl.found=true
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer

listeners=SASL_PLAINTEXT://localhost:9092
sasl.enabled.mechanisms=PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN
security.inter.broker.protocol=SASL_PLAINTEXT
super.users=User:admin

make jaas file, for example config/jaas_kafka_server.conf with lines

KafkaServer {
   org.apache.kafka.common.security.plain.PlainLoginModule required
   username="admin"
   password="admin-secret"
   user_admin="admin-secret"
   user_kafkaclient="client-secret";
};

Client {
       org.apache.zookeeper.server.auth.DigestLoginModule required
       username="zooclient"
       password="client-secret";
};

pass JVM a special param, in linux you can do it this way:

export KAFKA_OPTS="-Djava.security.auth.login.config=/home/nkm/apps/kafka_2.11-2.0.0/config/jaas_kafka_server.conf"

and run kafka broker

bin/kafka-server-start.sh config/server.properties

3. Kafka producer/consumer

make config/client.properties and add this

security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN

make jaas file, for example config/jaas_kafka_client.conf with lines

KafkaClient {
  org.apache.kafka.common.security.plain.PlainLoginModule required
  username="kafkaclient"
  password="client-secret";
};

pass JVM a special param, in linux you can do it this way:

export KAFKA_OPTS="-Djava.security.auth.login.config=/home/nkm/apps/kafka_2.11-2.0.0/config/jaas_kafka_client.conf"

and run producer

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test_topic --producer.config config/client.properties

run consumer

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test_topic --from-beginning --consumer.config config/client.properties 

So after all kafka broker connects to Zookeper using Client data in jaas file (it must correspond to zookeeper Server jaas data) and consumers and producers connect to broker using KafkaClient jaas data (it must correspond to KafkaServer authentication data)

And don't forget to replace "/home/nkm/apps/kafka_2.11-2.0.0" with your own kafka directory)

[Crim]

That worked a treat @nadeevkm Thanks a ton! I'll poke around this weekend and get this working in the webapp.

[Crim]

How does this UI look for setting up a cluster using SASL? Any suggestions or improvements?

Plain SASL authentication

GSSAPI / Custom Mechanism

Selecting GSSAPI or a Custom mechanism would basically just prompt you to enter your own JAAS configuration manually.

[thammuio]

Looks Great. We Use GSSAPI(Kerberos) with SSL as SASL_SSL. So It should also take path to trust store and its password in SSL Settings(when no ssl auth). Also, it should ask for Kafka security protocol options as - PLAINTEXT/SASL_PLAINTEXT/SASL_SSL/SSL.

[thammuio]

SASL JAAS Config (only applies to SASL based security) - used for yahoo kafka manager

com.sun.security.auth.module.Krb5LoginModule required doNotPrompt=true useTicketCache=false principal="user@HADOOP.COM" useKeyTab=true serviceName="kafka" keyTab="/home/user/user.keytab" client=true;

here is the jaas.conf file that clients uses

KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
doNotPrompt=true
useTicketCache=false
principal="user@HADOOP.COM"
useKeyTab=true
serviceName="kafka"
keyTab="/home/user/user.keytab"
client=true;
};

Client {
com.sun.security.auth.module.Krb5LoginModule required
doNotPrompt=true
useTicketCache=false
principal="user@HADOOP.COM"
useKeyTab=true
serviceName="kafka"
keyTab="/home/user/user.keytab"
client=true;
};

[Crim]

Just to make sure I'm understanding correctly....since the app already supports PLAINTEXT and SSL, when I add SASL support,

• PLAIN when you don't have SSL or SASL options enabled
• SSL when you have just the currently supported SSL options enabled.
• SASL_PLAIN when you do NOT have the currently supported SSL settings enabled, and have the new SASL settings enabled.
• SASL_SSL a combination of the existing SSL settings enabled, as well as the new SASL settings

This sounds correct?

[thammuio]

@Crim yes you are right.

One thing to point out is when we have SASL_SSL; we just use truststore(no ssl Auth - so keystore is not needed).

[Crim]

Good to know!

[Crim]

I've published a new release that should support this. Thanks!

[nadeevkm]

Thank you very much for your work! I'll check It in a few days!

[thammuio]

Thanks @Crim. when we have SASL auth, we dont need SSL auth; can you check on this; for SASL_SSL we just need SASL conf and SSL truststore.

[thammuio]

I am able to setup with SASL. I can use it. But, i am getting below error in Cluster Broker View:

Error: org.apache.kafka.common.errors.UnsupportedVersionException: The broker does not support DESCRIBE_CONFIGS

[Crim]

@thammuio I've created issue #115 for your error.

[Crim]

Sounds like when SASL is enabled, there's no need for the keystore and associated password?

[thammuio]

Thanks @Crim . Yes, when SASL is enabled no need of keystore details.

[Crim]

created issue #116 to deal with no longer requiring a KEYSTORE in this scenario and closing out this ticket.

Thanks!