Improve kerberos documents #4771

jiazhai · 2019-07-21T13:48:35Z

Improve kerberos documentation.

sijie · 2019-07-21T16:16:52Z

Anonymitaet · 2019-07-23T01:57:26Z

site2/docs/security-kerberos.md

 Note that if you are using Oracle Java, you need to download JCE policy files for your Java version and copy them to the `$JAVA_HOME/jre/lib/security` directory.

-#### Kerberos Principals
+Here is some other prerequisites that will use in this configuration.


Suggested change

Here is some other prerequisites that will use in this configuration.

Consider removing this sentence?

Reasons:

Based on the context, it does not need a transition sentence here.

If the sentence is removed, it has no impact on the doc.

Anonymitaet · 2019-07-23T01:57:54Z

site2/docs/security-kerberos.md

-#### Kerberos Principals
+Here is some other prerequisites that will use in this configuration.
+
+#### 1.1.1. Kerberos Principals


You want to use the ordered number list to show the hierarchy and structure cleary, that's good, but a more effective way is to generate the ordered number list automatically rather than manually.

seems hard to auto generate. removed them

Anonymitaet · 2019-07-23T02:04:04Z

site2/docs/security-kerberos.md

 Note that it is a *Kerberos* requirement that all your hosts can be resolved with their FQDNs.

-#### Configure how to connect to KDC
+The first part of Broker principal (e.g. `broker` in `broker/{hostname}@{REALM}`) is the `serverType` of each host, 


Suggested change

The first part of Broker principal (e.g. `broker` in `broker/{hostname}@{REALM}`) is the `serverType` of each host,

The first part of Broker principal (for example, `broker` in `broker/{hostname}@{REALM}`) is the `serverType` of each host,

Avoid using e.g. in a formal text.

Anonymitaet · 2019-07-23T02:18:15Z

site2/docs/security-kerberos.md


-#### Configure how to connect to KDC
+The first part of Broker principal (e.g. `broker` in `broker/{hostname}@{REALM}`) is the `serverType` of each host, 
+It can be value `broker` (host machine runs service Pulsar Broker) or `proxy` (host machine runs service Pulsar Proxy). 


What's the meaning of "It can be value"?

Anonymitaet · 2019-07-23T02:29:51Z

site2/docs/security-kerberos.md

    and the location of the keytab where the principal is stored. It allows the broker to use the keytab specified in this section.
 2. `PulsarClient` is a section name in the JASS file used by each client. This section tells the client which principal to use inside Kerberos
    and the location of the keytab where the principal is stored. It allows the client to use the keytab specified in this section.
+    In the following example, this `PulsarClient` section will also be used in the Pulsar internal admin configuration, 


Do you mean the following?

Suggested change

In the following example, this `PulsarClient` section will also be used in the Pulsar internal admin configuration,

In the following example, the `PulsarClient` section can be used in the Pulsar CLI tools as well, for example, `pulsar-client`, `pulsar-perf`, and `pulsar-admin`. You can add multiple `PulsarClient` sections for different purposes.

Anonymitaet · 2019-07-23T02:33:30Z

site2/docs/security-kerberos.md

+    in cli command of `bin/pulsar-client`, `bin/pulsar-perf` and `bin/pulsar-admin`. You can also add different section for different use case.

-It is also a choice to have 2 separate JAAS configuration files: the file for broker will only have `PulsarBroker` section; while the one for client only have `PulsarClient` section.
+It is also a choice to have 2 separate JAAS configuration files: the file for broker will have both section; while the one for client only have `PulsarClient` section.


Suggested change

It is also a choice to have 2 separate JAAS configuration files: the file for broker will have both section; while the one for client only have `PulsarClient` section.

You can have two separate JAAS configuration files:

* The file for a broker has both `PulsarBroker` and `PulsarClient` sections

* The file for a client only has `PulsarClient` section

Anonymitaet · 2019-07-23T02:35:02Z

site2/docs/security-kerberos.md

+
+#### 1.2.1. configure `broker.conf` file
+
+ In the `broker.conf` file, set Kerberos related configuration.


Suggested change

In the `broker.conf` file, set Kerberos related configuration.

In the `broker.conf` file, set Kerberos related configurations.

Anonymitaet · 2019-07-23T02:37:59Z

site2/docs/security-kerberos.md

+ - Set `saslJaasClientAllowedIds` regex for principal that is allowed to connect to broker;
+ - Set `saslJaasBrokerSectionName` that corresponding to the section in JAAS configuration file for broker;

+ In order to get Pulsar internal admin client working right, You need these 2 configuration in `broker.conf` file. 


Suggested change

In order to get Pulsar internal admin client working right, You need these 2 configuration in `broker.conf` file.

To make Pulsar internal admin client work properly, you need to set the configurations in the `broker.conf` file as below.

Anonymitaet · 2019-07-23T02:38:14Z

site2/docs/security-kerberos.md

+ - Set `saslJaasBrokerSectionName` that corresponding to the section in JAAS configuration file for broker;

+ In order to get Pulsar internal admin client working right, You need these 2 configuration in `broker.conf` file. 
+ - set `brokerClientAuthenticationPlugin` to client plugin `AuthenticationSasl`;


Suggested change

- set `brokerClientAuthenticationPlugin` to client plugin `AuthenticationSasl`;

- Set `brokerClientAuthenticationPlugin` to client plugin `AuthenticationSasl`;

Anonymitaet · 2019-07-23T02:41:12Z

site2/docs/security-kerberos.md


+ In order to get Pulsar internal admin client working right, You need these 2 configuration in `broker.conf` file. 
+ - set `brokerClientAuthenticationPlugin` to client plugin `AuthenticationSasl`;
+ - set `brokerClientAuthenticationParameters` to value in Json string `{"saslJaasClientSectionName":"PulsarClient", "serverType":"broker"}`, in which `PulsarClient` is the section name in above `pulsar_jaas.conf` file, and `"serverType":"broker"` indicate that internal admin client will connect to a Pulsar Broker;


Suggested change

- set `brokerClientAuthenticationParameters` to value in Json string `{"saslJaasClientSectionName":"PulsarClient", "serverType":"broker"}`, in which `PulsarClient` is the section name in above `pulsar_jaas.conf` file, and `"serverType":"broker"` indicate that internal admin client will connect to a Pulsar Broker;

- Set `brokerClientAuthenticationParameters` to a value in a JSON string `{"saslJaasClientSectionName":"PulsarClient", "serverType":"broker"}`, where `PulsarClient` is the section name in the `pulsar_jaas.conf` file, and `"serverType":"broker"` indicates the internal admin client will connect to a Pulsar Broker;

Anonymitaet · 2019-07-23T02:42:22Z

site2/docs/security-kerberos.md

-2. Set JVM parameter for JAAS configuration file and krb5 configuration file with additional option.
+#### 1.2.2. Set Broker JVM parameter
+
+ Set JVM parameter for JAAS configuration file and krb5 configuration file with additional option.


Suggested change

Set JVM parameter for JAAS configuration file and krb5 configuration file with additional option.

Set JVM parameters for JAAS configuration file and krb5 configuration file with additional options.

Anonymitaet · 2019-07-23T02:46:55Z

site2/docs/security-kerberos.md


-## Kerberos configuration for working with Pulsar Proxy
+You can add this at the end of `PULSAR_EXTRA_OPTS` in the file [`pulsar_tools_env.sh`](https://github.com/apache/pulsar/blob/master/conf/pulsar_tools_env.sh),
+Or you could add this line `OPTS="$OPTS -Djava.security.auth.login.config=/etc/pulsar/pulsar_jaas.conf -Djava.security.krb5.conf=/etc/pulsar/krb5.conf "` directly in to the CLI tools script.


Suggested change

Or you could add this line `OPTS="$OPTS -Djava.security.auth.login.config=/etc/pulsar/pulsar_jaas.conf -Djava.security.krb5.conf=/etc/pulsar/krb5.conf "` directly in to the CLI tools script.

or add this line `OPTS="$OPTS -Djava.security.auth.login.config=/etc/pulsar/pulsar_jaas.conf -Djava.security.krb5.conf=/etc/pulsar/krb5.conf "` directly to the CLI tool script.

Anonymitaet · 2019-07-23T02:47:27Z

site2/docs/security-kerberos.md

+You can add this at the end of `PULSAR_EXTRA_OPTS` in the file [`pulsar_tools_env.sh`](https://github.com/apache/pulsar/blob/master/conf/pulsar_tools_env.sh),
+Or you could add this line `OPTS="$OPTS -Djava.security.auth.login.config=/etc/pulsar/pulsar_jaas.conf -Djava.security.krb5.conf=/etc/pulsar/krb5.conf "` directly in to the CLI tools script.
+
+The meaning of configurations are the same as in Java client section.


Suggested change

The meaning of configurations are the same as in Java client section.

The meaning of configurations is the same as that in the Java client section.

Anonymitaet · 2019-07-23T02:50:46Z

site2/docs/security-kerberos.md

+requireClientAuthScheme=sasl
+```
+
+And add a section of `Client` configurations, which use in the `pulsar_jaas.conf` that used by Pulsar Broker:


What's the meaning of "which use in"?

Do you mean "which is used in"?

Anonymitaet · 2019-07-23T02:51:14Z

site2/docs/security-kerberos.md

+
+## 5. Regarding authentication between BookKeeper and Broker
+
+Pulsar Broker acts as a Kerberos client when authenticating with Bookie. According to [BookKeeper document](http://bookkeeper.apache.org/docs/latest/security/sasl/), you need add `bookkeeperClientAuthenticationPlugin` parameter in `broker.conf`:


Suggested change

Pulsar Broker acts as a Kerberos client when authenticating with Bookie. According to [BookKeeper document](http://bookkeeper.apache.org/docs/latest/security/sasl/), you need add `bookkeeperClientAuthenticationPlugin` parameter in `broker.conf`:

Pulsar Broker acts as a Kerberos client when authenticating with Bookie. According to [BookKeeper document](http://bookkeeper.apache.org/docs/latest/security/sasl/), you need to add `bookkeeperClientAuthenticationPlugin` parameter in `broker.conf`:

Anonymitaet · 2019-07-23T02:51:46Z

site2/docs/security-kerberos.md

 ```

-For more details of how to configure Kerberos for BookKeeper and Zookeeper, refer to [BookKeeper document](http://bookkeeper.apache.org/docs/latest/security/sasl/).
+In this setting, `SASLClientProviderFactory` will create a BookKeeper SASL client in Broker, and use this client authenticate with Bookie node.


Suggested change

In this setting, `SASLClientProviderFactory` will create a BookKeeper SASL client in Broker, and use this client authenticate with Bookie node.

In this setting, `SASLClientProviderFactory` will create a BookKeeper SASL client in Broker, and use this client to authenticate with a Bookie node.

Who uses this client?

What is "this client"? SASLClientProviderFactory?

Anonymitaet · 2019-07-23T02:56:34Z

I see you made the same changes in two files, please remember to check all similar cases.

jiazhai · 2019-07-25T07:11:43Z

@Anonymitaet updated.

sijie · 2019-07-26T04:31:12Z

@Anonymitaet please take a look.

// run cpp tests

sijie · 2019-07-26T04:31:24Z

run integration tests

jiazhai · 2019-07-29T04:24:18Z

run integration tests

jiazhai · 2019-07-29T12:05:17Z

run integration tests

Anonymitaet · 2019-07-29T13:25:00Z

site2/docs/security-kerberos.md

-2. Set JVM parameter for JAAS configuration file and krb5 configuration file with additional option.
+
+
+Step 2.  Set JVM parameters for JAAS configuration file and krb5 configuration file with additional options.


Suggested change

Step 2. Set JVM parameters for JAAS configuration file and krb5 configuration file with additional options.

Step 2. Set JVM parameters for JAAS configuration file and krb5 configuration file with additional options.

Anonymitaet · 2019-07-29T13:25:05Z

site2/docs/security-kerberos.md


-If you are using command line, you can continue with these step:
-1. Config your `client.conf`: 
+#### configure CLI tools


Suggested change

#### configure CLI tools

#### Configure CLI tools

Anonymitaet · 2019-07-29T13:26:19Z

site2/website/versioned_docs/version-2.4.0/security-kerberos.md

    and the location of the keytab where the principal is stored. It allows the broker to use the keytab specified in this section.
 2. `PulsarClient` is a section name in the JASS file used by each client. This section tells the client which principal to use inside Kerberos
    and the location of the keytab where the principal is stored. It allows the client to use the keytab specified in this section.
+    In the following example, this `PulsarClient` section will also be reused in both the Pulsar internal admin configuration and in cli command of `bin/pulsar-client`, `bin/pulsar-perf` and `bin/pulsar-admin`. You can also add different section for different use case.


Suggested change

In the following example, this `PulsarClient` section will also be reused in both the Pulsar internal admin configuration and in cli command of `bin/pulsar-client`, `bin/pulsar-perf` and `bin/pulsar-admin`. You can also add different section for different use case.

In the following example, this `PulsarClient` section will also be reused in both the Pulsar internal admin configuration and in CLI command of `bin/pulsar-client`, `bin/pulsar-perf` and `bin/pulsar-admin`. You can also add different sections for different use cases.

Anonymitaet · 2019-07-29T13:26:51Z

site2/website/versioned_docs/version-2.4.0/security-kerberos.md

+    In the following example, this `PulsarClient` section will also be reused in both the Pulsar internal admin configuration and in cli command of `bin/pulsar-client`, `bin/pulsar-perf` and `bin/pulsar-admin`. You can also add different section for different use case.
+
+You can have 2 separate JAAS configuration files: 
+* the file for broker will have sections of both `PulsarBroker` and `PulsarClient`; 


Suggested change

* the file for broker will have sections of both `PulsarBroker` and `PulsarClient`;

* the file for a broker has sections of both `PulsarBroker` and `PulsarClient`;

Anonymitaet · 2019-07-29T13:27:11Z

site2/website/versioned_docs/version-2.4.0/security-kerberos.md

+
+You can have 2 separate JAAS configuration files: 
+* the file for broker will have sections of both `PulsarBroker` and `PulsarClient`; 
+* the file for client only have `PulsarClient` section.


Suggested change

* the file for client only have `PulsarClient` section.

* the file for a client only has a `PulsarClient` section.

Anonymitaet · 2019-07-29T13:28:50Z

site2/website/versioned_docs/version-2.4.0/security-kerberos.md

 saslJaasClientAllowedIds=.*client.*
 saslJaasBrokerSectionName=PulsarBroker
+
+## Authentication settings of the broker itself. Used when the broker connects to other brokers


The heading (##) is too long?

This is not head, it is comments in shell

Anonymitaet · 2019-07-29T13:29:14Z

site2/website/versioned_docs/version-2.4.0/security-kerberos.md


-If you are using command line, you can continue with these step:
-1. Config your `client.conf`: 
+#### configure CLI tools


Suggested change

#### configure CLI tools

#### Configure CLI tools

Anonymitaet · 2019-07-29T13:29:43Z

site2/website/versioned_docs/version-2.4.0/security-kerberos.md

-2. Set JVM parameter for JAAS configuration file and krb5 configuration file with additional option.
+
+
+Step 2.  Set JVM parameters for JAAS configuration file and krb5 configuration file with additional options.


Suggested change

Step 2. Set JVM parameters for JAAS configuration file and krb5 configuration file with additional options.

Step 2. Set JVM parameters for JAAS configuration file and krb5 configuration file with additional options.

Anonymitaet · 2019-07-29T13:30:18Z

site2/website/versioned_docs/version-2.4.0/security-kerberos.md


-## Kerberos configuration for working with Pulsar Proxy
+You can add this at the end of `PULSAR_EXTRA_OPTS` in the file [`pulsar_tools_env.sh`](https://github.com/apache/pulsar/blob/master/conf/pulsar_tools_env.sh),
+Or add this line `OPTS="$OPTS -Djava.security.auth.login.config=/etc/pulsar/pulsar_jaas.conf -Djava.security.krb5.conf=/etc/pulsar/krb5.conf "` directly in to the CLI tool script.


Suggested change

Or add this line `OPTS="$OPTS -Djava.security.auth.login.config=/etc/pulsar/pulsar_jaas.conf -Djava.security.krb5.conf=/etc/pulsar/krb5.conf "` directly in to the CLI tool script.

or add this line `OPTS="$OPTS -Djava.security.auth.login.config=/etc/pulsar/pulsar_jaas.conf -Djava.security.krb5.conf=/etc/pulsar/krb5.conf "` directly to the CLI tool script.

Anonymitaet · 2019-07-29T13:31:20Z

site2/website/versioned_docs/version-2.4.0/security-kerberos.md

 ```

-For more details of how to configure Kerberos for BookKeeper and Zookeeper, refer to [BookKeeper document](http://bookkeeper.apache.org/docs/latest/security/sasl/).
+In this setting, `SASLClientProviderFactory` will create a BookKeeper SASL client in Broker, and Broker use created SASL client to authenticate with a Bookie node.


Suggested change

In this setting, `SASLClientProviderFactory` will create a BookKeeper SASL client in Broker, and Broker use created SASL client to authenticate with a Bookie node.

In this setting, `SASLClientProviderFactory` creates a BookKeeper SASL client in a broker, and the broker uses the created SASL client to authenticate with a bookie node.

jiazhai · 2019-07-30T02:37:49Z

@Anonymitaet updated

jiazhai · 2019-07-30T07:16:35Z

run cpp tests

jiazhai · 2019-07-31T00:55:46Z

run java8 tests

jiazhai · 2019-08-01T01:29:52Z

run java8 tests

jiazhai · 2019-08-02T03:51:45Z

run java8 tests

sijie · 2019-08-02T04:07:28Z

// Couldn?t find any executable in /home/jenkins/tools/maven/apache-maven-3.5.0

run java8 tests

jiazhai added 2 commits July 21, 2019 19:16

improve doc for kerberos

d047270

add more info

25d4117

jiazhai added the doc Your PR contains doc changes, no matter whether the changes are in markdown or code files. label Jul 21, 2019

jiazhai self-assigned this Jul 21, 2019

sijie approved these changes Jul 21, 2019

View reviewed changes

sijie added the area/security label Jul 21, 2019

sijie added this to the 2.5.0 milestone Jul 21, 2019

also update 2.4.0 doc

7427724

Anonymitaet reviewed Jul 23, 2019

View reviewed changes

updated following comments

e35d2c4

Anonymitaet reviewed Jul 29, 2019

View reviewed changes

change following comments

878ed18

Anonymitaet approved these changes Jul 30, 2019

View reviewed changes

sijie merged commit 710b4b1 into apache:master Aug 2, 2019

	The first part of Broker principal (e.g. `broker` in `broker/{hostname}@{REALM}`) is the `serverType` of each host,
	The first part of Broker principal (for example, `broker` in `broker/{hostname}@{REALM}`) is the `serverType` of each host,

	In the following example, this `PulsarClient` section will also be used in the Pulsar internal admin configuration,
	In the following example, the `PulsarClient` section can be used in the Pulsar CLI tools as well, for example, `pulsar-client`, `pulsar-perf`, and `pulsar-admin`. You can add multiple `PulsarClient` sections for different purposes.

-It is also a choice to have 2 separate JAAS configuration files: the file for broker will have both section; while the one for client only have `PulsarClient` section.
+You can have two separate JAAS configuration files:
+* The file for a broker has both `PulsarBroker` and `PulsarClient` sections
+* The file for a client only has `PulsarClient` section


		#### 1.2.1. configure `broker.conf` file

		In the `broker.conf` file, set Kerberos related configuration.

	In order to get Pulsar internal admin client working right, You need these 2 configuration in `broker.conf` file.
	To make Pulsar internal admin client work properly, you need to set the configurations in the `broker.conf` file as below.

	- set `brokerClientAuthenticationPlugin` to client plugin `AuthenticationSasl`;
	- Set `brokerClientAuthenticationPlugin` to client plugin `AuthenticationSasl`;

	- set `brokerClientAuthenticationParameters` to value in Json string `{"saslJaasClientSectionName":"PulsarClient", "serverType":"broker"}`, in which `PulsarClient` is the section name in above `pulsar_jaas.conf` file, and `"serverType":"broker"` indicate that internal admin client will connect to a Pulsar Broker;
	- Set `brokerClientAuthenticationParameters` to a value in a JSON string `{"saslJaasClientSectionName":"PulsarClient", "serverType":"broker"}`, where `PulsarClient` is the section name in the `pulsar_jaas.conf` file, and `"serverType":"broker"` indicates the internal admin client will connect to a Pulsar Broker;

	Set JVM parameter for JAAS configuration file and krb5 configuration file with additional option.
	Set JVM parameters for JAAS configuration file and krb5 configuration file with additional options.

	Or you could add this line `OPTS="$OPTS -Djava.security.auth.login.config=/etc/pulsar/pulsar_jaas.conf -Djava.security.krb5.conf=/etc/pulsar/krb5.conf "` directly in to the CLI tools script.
	or add this line `OPTS="$OPTS -Djava.security.auth.login.config=/etc/pulsar/pulsar_jaas.conf -Djava.security.krb5.conf=/etc/pulsar/krb5.conf "` directly to the CLI tool script.

	The meaning of configurations are the same as in Java client section.
	The meaning of configurations is the same as that in the Java client section.


		## 5. Regarding authentication between BookKeeper and Broker

		Pulsar Broker acts as a Kerberos client when authenticating with Bookie. According to [BookKeeper document](http://bookkeeper.apache.org/docs/latest/security/sasl/), you need add `bookkeeperClientAuthenticationPlugin` parameter in `broker.conf`:

	In this setting, `SASLClientProviderFactory` will create a BookKeeper SASL client in Broker, and use this client authenticate with Bookie node.
	In this setting, `SASLClientProviderFactory` will create a BookKeeper SASL client in Broker, and use this client to authenticate with a Bookie node.

		2. Set JVM parameter for JAAS configuration file and krb5 configuration file with additional option.


		Step 2. Set JVM parameters for JAAS configuration file and krb5 configuration file with additional options.

	* the file for broker will have sections of both `PulsarBroker` and `PulsarClient`;
	* the file for a broker has sections of both `PulsarBroker` and `PulsarClient`;

	* the file for client only have `PulsarClient` section.
	* the file for a client only has a `PulsarClient` section.

Improve kerberos documents #4771

Improve kerberos documents #4771

Uh oh!

Conversation

jiazhai commented Jul 21, 2019

Uh oh!

sijie commented Jul 21, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Anonymitaet commented Jul 23, 2019

Uh oh!

jiazhai commented Jul 25, 2019

Uh oh!

sijie commented Jul 26, 2019

Uh oh!

sijie commented Jul 26, 2019

Uh oh!

jiazhai commented Jul 29, 2019

Uh oh!

jiazhai commented Jul 29, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jiazhai Jul 30, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jiazhai commented Jul 30, 2019

Uh oh!

jiazhai Jul 30, 2019 •

edited

Loading