From a062c440751088bb5b3ea9df1cfa90136e820fce Mon Sep 17 00:00:00 2001 From: Nick Allen Date: Wed, 5 Apr 2017 16:55:42 -0400 Subject: [PATCH 1/3] METRON-829 Use Fastcapa with Kerberos --- metron-sensors/fastcapa/README.md | 78 +++++++++++++++++++++++++++++++ 1 file changed, 78 insertions(+) diff --git a/metron-sensors/fastcapa/README.md b/metron-sensors/fastcapa/README.md index 74e7809e2a..eb26e2661e 100644 --- a/metron-sensors/fastcapa/README.md +++ b/metron-sensors/fastcapa/README.md @@ -9,6 +9,9 @@ Fastcapa leverages the Data Plane Development Kit ([DPDK](http://dpdk.org/)). D * [Requirements](#requirements) * [Installation](#installation) * [Usage](#usage) + * [Parameters](#parameters) + * [Output](#output) + * [Kerberos](#kerberos) * [How It Works](#how-it-works) * [Performance](#performance) * [FAQs](#faqs) @@ -181,6 +184,7 @@ The probe has been tested with [Librdkafka 0.9.4](https://github.com/edenhill/li cd incubator-metron/metron-sensors/fastcapa make ``` + Usage ----- @@ -316,6 +320,80 @@ When running the probe some basic counters are output to stdout. Of course duri * `[kaf]` + `out`: A total of 7 packets has successfully reached Kafka. * `[kaf]` + `queued`: There is 1 packet within the `rdkafka` queue waiting to be sent. +### Kerberos + +The probe can be used in a Kerberized environment. The following additional steps make the following assumptions about the Kerberized environment. These assumptions may need altered to fit your environment. + +* The Kafka broker is at "kafka1:6667" +* Zookeeper is at "zookeeper1:2181" +* The Kafka security protocol is "SASL_PLAINTEXT" +* The keytab used is located at `/etc/security/keytabs/metron.headless.keytab` +* The service principal is "metron@EXAMPLE.COM" + +1. Install [Cyrus SASL](http://www.cyrusimap.org/sasl/index.html#sasl-index). + ``` + yum install -y cyrus-sasl cyrus-sasl-devel cyrus-sasl-gssapi + ``` + +1. Kerberos is probably already installed. + ``` + yum -y install krb5-server krb5-libs krb5-workstation + ``` + +1. Build Librdkafka with SASL support (` --enable-sasl`). + ``` + wget https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz -O - | tar -xz + cd librdkafka-0.9.4/ + ./configure --prefix=$RDK_PREFIX --enable-sasl + make + make install + ``` + +1. Validate Librdkafka does indeed support SASL. Run the following command and ensure that SASL is returned. + ``` + $ examples/rdkafka_example -X builtin.features + builtin.features = gzip,snappy,ssl,sasl,regex + ``` + +1. Create a JAAS configuration file at `~/.java.login.config` + ``` + $ cat ~/.java.login.config + KafkaClient { + com.sun.security.auth.module.Krb5LoginModule required + useTicketCache=false + useKeyTab=true + principal="metron@EXAMPLE.COM" + keyTab="/etc/security/keytabs/metron.headless.keytab" + renewTicket=true + debug=true + serviceName="kafka" + storeKey=true; + }; + ``` +1. Let your Java environment know where it can find the JAAS configuration file. Edit the file at `$JAVA_HOME/jre/lib/security/java.security` and add the line below. + ``` + login.config.url.1=file:${user.home}/.java.login.config + ``` + +1. Grant access to your Kafka topic. In this example, it is simply named "pcap". + ``` + $KAFKA_HOME/bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=zookeeper1:2181 --add --allow-principal User:metron --topic pcap + ``` + +1. Obtain a Kerberos ticket. + ``` + kinit -kt /etc/security/keytabs/metron.headless.keytab metron@EXAMPLE.COM + ``` + +1. Add the following additional configuration values to your Fastcapa configuration file. + ``` + security.protocol = SASL_PLAINTEXT + sasl.kerberos.keytab = /etc/security/keytabs/metron.headless.keytab + sasl.kerberos.principal = metron@EXAMPLE.COM + ``` + +1. Now run Fastcapa as you normally would. It should have no problem landing packets in your kerberized Kafka broker. + How It Works ------ From 3165e5f4c5aad51560be966a26b97aae65ef1c7d Mon Sep 17 00:00:00 2001 From: Nick Allen Date: Wed, 5 Apr 2017 17:08:53 -0400 Subject: [PATCH 2/3] Grammar is king --- metron-sensors/fastcapa/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/metron-sensors/fastcapa/README.md b/metron-sensors/fastcapa/README.md index eb26e2661e..0863ff648f 100644 --- a/metron-sensors/fastcapa/README.md +++ b/metron-sensors/fastcapa/README.md @@ -322,7 +322,7 @@ When running the probe some basic counters are output to stdout. Of course duri ### Kerberos -The probe can be used in a Kerberized environment. The following additional steps make the following assumptions about the Kerberized environment. These assumptions may need altered to fit your environment. +The probe can be used in a Kerberized environment. Follow these additional steps to use Fastcapa with Kerberos. The following assumptions have been made. These may need altered to fit your environment. * The Kafka broker is at "kafka1:6667" * Zookeeper is at "zookeeper1:2181" From d9575a3481cb5cda4217a8b8d68590109e9c6c6b Mon Sep 17 00:00:00 2001 From: Nick Allen Date: Mon, 10 Apr 2017 10:37:01 -0400 Subject: [PATCH 3/3] Removed JAAS configuration steps which are not needed with Fastcapa. --- metron-sensors/fastcapa/README.md | 45 ++++++++----------------------- 1 file changed, 11 insertions(+), 34 deletions(-) diff --git a/metron-sensors/fastcapa/README.md b/metron-sensors/fastcapa/README.md index 0863ff648f..5efdb3f5a4 100644 --- a/metron-sensors/fastcapa/README.md +++ b/metron-sensors/fastcapa/README.md @@ -324,21 +324,11 @@ When running the probe some basic counters are output to stdout. Of course duri The probe can be used in a Kerberized environment. Follow these additional steps to use Fastcapa with Kerberos. The following assumptions have been made. These may need altered to fit your environment. -* The Kafka broker is at "kafka1:6667" -* Zookeeper is at "zookeeper1:2181" -* The Kafka security protocol is "SASL_PLAINTEXT" +* The Kafka broker is at `kafka1:6667` +* Zookeeper is at `zookeeper1:2181` +* The Kafka security protocol is `SASL_PLAINTEXT` * The keytab used is located at `/etc/security/keytabs/metron.headless.keytab` -* The service principal is "metron@EXAMPLE.COM" - -1. Install [Cyrus SASL](http://www.cyrusimap.org/sasl/index.html#sasl-index). - ``` - yum install -y cyrus-sasl cyrus-sasl-devel cyrus-sasl-gssapi - ``` - -1. Kerberos is probably already installed. - ``` - yum -y install krb5-server krb5-libs krb5-workstation - ``` +* The service principal is `metron@EXAMPLE.COM` 1. Build Librdkafka with SASL support (` --enable-sasl`). ``` @@ -349,35 +339,22 @@ The probe can be used in a Kerberized environment. Follow these additional step make install ``` -1. Validate Librdkafka does indeed support SASL. Run the following command and ensure that SASL is returned. +1. Validate Librdkafka does indeed support SASL. Run the following command and ensure that `sasl` is returned as a built-in feature. ``` $ examples/rdkafka_example -X builtin.features builtin.features = gzip,snappy,ssl,sasl,regex ``` -1. Create a JAAS configuration file at `~/.java.login.config` - ``` - $ cat ~/.java.login.config - KafkaClient { - com.sun.security.auth.module.Krb5LoginModule required - useTicketCache=false - useKeyTab=true - principal="metron@EXAMPLE.COM" - keyTab="/etc/security/keytabs/metron.headless.keytab" - renewTicket=true - debug=true - serviceName="kafka" - storeKey=true; - }; + If it is not, ensure that you have `libsasl` or `libsasl2` installed. On CentOS, this can be installed with the following command. ``` -1. Let your Java environment know where it can find the JAAS configuration file. Edit the file at `$JAVA_HOME/jre/lib/security/java.security` and add the line below. - ``` - login.config.url.1=file:${user.home}/.java.login.config + yum install -y cyrus-sasl cyrus-sasl-devel cyrus-sasl-gssapi ``` -1. Grant access to your Kafka topic. In this example, it is simply named "pcap". +1. Grant access to your Kafka topic. In this example, it is simply named `pcap`. ``` - $KAFKA_HOME/bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=zookeeper1:2181 --add --allow-principal User:metron --topic pcap + $KAFKA_HOME/bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer \ + --authorizer-properties zookeeper.connect=zookeeper1:2181 \ + --add --allow-principal User:metron --topic pcap ``` 1. Obtain a Kerberos ticket.