Support for TLS/SSL and SASL-security broker connections

[ekoutanov]

Kafdrop supports TLS (SSL) and SASL connections for encryption and authentication. This can be configured by providing a combination of the following files (placed into the Kafka root directory):

• kafka.truststore.jks: specifying the certificate for authenticating brokers, if TLS is enabled.
• kafka.keystore.jks: specifying the private key to authenticate the client to the broker, if mutual TLS authentication is required.
• kafka.properties: specifying the necessary configuration, including key/truststore passwords, cipher suites, enabled TLS protocol versions, username/password pairs, etc. When supplying the truststore and/or keystore files, the ssl.truststore.location and ssl.keystore.location properties will be assigned automatically.

Using Docker

The three files above can be supplied to a Docker instance in base-64-encoded form via environment variables:

docker run -d --rm -p 9000:9000 \
    -e ZOOKEEPER_CONNECT=<host:port,host:port> \
    -e KAFKA_BROKERCONNECT=<host:port,host:port> \
    -e KAFKA_PROPERTIES=$(cat kafka.properties | base64) \
    -e KAFKA_TRUSTSTORE=$(cat kafka.truststore.jks | base64) \   # optional
    -e KAFKA_KEYSTORE=$(cat kafka.keystore.jks | base64)         # optional
    obsidiandynamics/kafdrop

Using Helm

Like in the Docker example, supply the files in base-64 form:

helm upgrade -i kafdrop chart --set image.tag=3.x.x \
    --set zookeeper.connect=<host:port,host:port> \
    --set kafka.brokerConnect=<host:port,host:port> \
    --set kafka.properties="$(cat kafka.properties | base64)" \
    --set kafka.truststore="$(cat kafka.truststore | base64)" \
    --set kafka.keystore="$(cat kafka.keystore | base64)"

Closes #28

[adivardhan]

Kafdrop supports TLS (SSL) and SASL connections for encryption and authentication. This can be configured by providing a combination of the following files (placed into the Kafka root directory):

• kafka.truststore.jks: specifying the certificate for authenticating brokers, if TLS is enabled.
• kafka.keystore.jks: specifying the private key to authenticate the client to the broker, if mutual TLS authentication is required.
• kafka.properties: specifying the necessary configuration, including key/truststore passwords, cipher suites, enabled TLS protocol versions, username/password pairs, etc. When supplying the truststore and/or keystore files, the ssl.truststore.location and ssl.keystore.location properties will be assigned automatically.

Using Docker

The three files above can be supplied to a Docker instance in base-64-encoded form via environment variables:

docker run -d --rm -p 9000:9000 \
    -e ZOOKEEPER_CONNECT=<host:port,host:port> \
    -e KAFKA_BROKERCONNECT=<host:port,host:port> \
    -e KAFKA_PROPERTIES=$(cat kafka.properties | base64) \
    -e KAFKA_TRUSTSTORE=$(cat kafka.truststore.jks | base64) \   # optional
    -e KAFKA_KEYSTORE=$(cat kafka.keystore.jks | base64)         # optional
    obsidiandynamics/kafdrop

Using Helm

Like in the Docker example, supply the files in base-64 form:

helm upgrade -i kafdrop chart --set image.tag=3.x.x \
    --set zookeeper.connect=<host:port,host:port> \
    --set kafka.brokerConnect=<host:port,host:port> \
    --set kafka.properties="$(cat kafka.properties | base64)" \
    --set kafka.truststore="$(cat kafka.truststore | base64)" \
    --set kafka.keystore="$(cat kafka.keystore | base64)"

Closes #28

When you say "combination of the following files (placed into the Kafka root directory):", I'm assuming you mean we have to place a keystore, truststore and a custom kafka.properties file in the root of Kafdrop (in its classpath)?

[ekoutanov]

@adivardhan only for the scenario when you're running from a Jar. In this case they need to be in the same directory as Kafdrop.

When running in Docker or Kubernetes, you just need to pass in the base-64-encoded contents of these files. Where they reside is irrelevant.

[adivardhan]

@adivardhan only for the scenario when you're running from a Jar. In this case they need to be in the same directory as Kafdrop.

When running in Docker or Kubernetes, you just need to pass in the base-64-encoded contents of these files. Where they reside is irrelevant.

I'm fairly new to the security setup.
The keystore & truststore are the certificates of the Kafka broker, right? (Or the machine where the broker is running).
These files can be generated by using keytool utility? Similar to how confluent docs and kafka docs have mentioned?
Then we place the same files here in Kafdrop (if running via jar)?

[MrMoronIV]

And how to set the correct settings when using the jar file as a systemd instance?

/kafdrop/target/kafka.properties

security.protocol=SASL_SSL
ssl.endpoint.identification.algorithm=
ssl.truststore.password=blablabla
sasl.mechanism=SCRAM-SHA-256
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required
    username="kafkabroker" password="blablabla";

I currently get this error:
[AdminClient clientId=adminclient-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Which makes sense since that port is not active, SASL_SSL is active on port 9094, where
and how would I define that setting? The truststore is in /kafdrop/target/ next to the jar

EDIT:
sorry, the first step is obvious:

/root/kafdrop/target/kafdrop-3.14.0-SNAPSHOT.jar --server.port=9001 --management.server.port=9001 --kafka.brokerConnect=kafka1.domain.com:9094,kafka2.domain.com:9094,kafka3.domain.com:9094