Skip to content

Commit

Permalink
Merge recent docs changes in apache#16891, apache#16941, and apache#1…
Browse files Browse the repository at this point in the history 
…7004
  • Loading branch information
@momo-jun
momo-jun committed Aug 9, 2022
1 parent eb12bb4 commit d4cb371
Show file tree
Hide file tree
Showing 6 changed files with 244 additions and 101 deletions.
2 changes: 1 addition & 1 deletion site2/website/versioned_docs/version-2.10.x/client-libraries-java.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@ You can set the client memory allocator configurations through Java properties.<
|---|---|---|---|---
`pulsar.allocator.pooled` | String | If set to `true`, the client uses a direct memory pool. <br /> If set to `false`, the client uses a heap memory without pool | true | <li> true </li> <li> false </li>
`pulsar.allocator.exit_on_oom` | String | Whether to exit the JVM when OOM happens | false | <li> true </li> <li> false </li>
`pulsar.allocator.leak_detection` | String | Service URL provider for Pulsar service | Disabled | <li> Disabled </li> <li> Simple </li> <li> Advanced </li> <li> Paranoid </li>
`pulsar.allocator.leak_detection` | String | The leak detection policy for Pulsar bytebuf allocator. <li> **Disabled**: No leak detection and no overhead. </li> <li> **Simple**: Instruments 1% of the allocated buffer to track for leaks. </li> <li> **Advanced**: Instruments 1% of the allocated buffer to track for leaks, reporting stack traces of places where the buffer is used. </li> <li> **Paranoid**: Instruments 100% of the allocated buffer to track for leaks, reporting stack traces of places where the buffer is used and introduces a significant overhead. </li> | Disabled | <li> Disabled </li> <li> Simple </li> <li> Advanced </li> <li> Paranoid </li>
`pulsar.allocator.out_of_memory_policy` | String | When an OOM occurs, the client throws an exception or fallbacks to heap | FallbackToHeap | <li> ThrowException </li> <li> FallbackToHeap </li>

**Example**:
Expand Down
30 changes: 30 additions & 0 deletions site2/website/versioned_docs/version-2.10.x/io-debezium-source.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ The configuration of Debezium source connector has the following properties.
| `database.history.pulsar.service.url` | true | null | Pulsar cluster service URL for history topic. |
| `offset.storage.topic` | true | null | Record the last committed offsets that the connector successfully completes. |
| `json-with-envelope` | false | false | Present the message only consist of payload.
| `database.history.pulsar.reader.config` | false | null | The configs of the reader for the database schema history topic, in the form of a JSON string with key-value pairs. <br />**Note:** This property is only available in 2.10.2 and later versions. |
| `offset.storage.reader.config` | false | null | The configs of the reader for the kafka connector offsets topic, in the form of a JSON string with key-value pairs. <br />**Note:** This property is only available in 2.10.2 and later versions.|

### Converter Options

Expand All @@ -55,7 +57,35 @@ Schema.AUTO_CONSUME(), KeyValueEncodingType.SEPARATED)`, and the message consist
| `mongodb.password` | true | null | Password to be used when connecting to MongoDB. This is required only when MongoDB is configured to use authentication. |
| `mongodb.task.id` | true | null | The taskId of the MongoDB connector that attempts to use a separate task for each replica set. |

### Customize the Reader config for the metadata topics

:::note

The customization is only available in 2.10.2 and later versions.

:::

The Debezium Connector exposes `database.history.pulsar.reader.config` and `offset.storage.reader.config` to configure the reader of database schema history topic and the Kafka connector offsets topic. For example, it can be used to configure the subscription name and other reader configurations. You can find the available configurations at [ReaderConfigurationData](https://github.com/apache/pulsar/blob/master/pulsar-client/src/main/java/org/apache/pulsar/client/impl/conf/ReaderConfigurationData.java).

For example, to configure the subscription name for both Readers, you can add the following configuration:
* JSON

```json
{
"configs": {
"database.history.pulsar.reader.config": "{\"subscriptionName\":\"history-reader\"}",
"offset.storage.reader.config": "{\"subscriptionName\":\"offset-reader\"}",
}
}
```

* YAML

```yaml
configs:
database.history.pulsar.reader.config: "{\"subscriptionName\":\"history-reader\"}"
offset.storage.reader.config: "{\"subscriptionName\":\"offset-reader\"}"
```

## Example of MySQL

Expand Down
94 changes: 61 additions & 33 deletions site2/website/versioned_docs/version-2.10.x/security-basic-auth.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,9 @@ Install [`htpasswd`](https://httpd.apache.org/docs/2.4/programs/htpasswd.html) i
## Create your authentication file

:::note

Currently, you can use MD5 (recommended) and CRYPT encryption to authenticate your password.

:::

Create a password file named `.htpasswd` with a user account `superuser/admin`:
Expand All @@ -55,53 +57,63 @@ superuser:$apr1$GBIYZYFZ$MzLcPrvoUky16mLcK6UtX/

## Enable basic authentication on brokers

To configure brokers to authenticate clients, complete the following steps.

1. Add the following parameters to the `conf/broker.conf` file. If you use a standalone Pulsar, you need to add these parameters to the `conf/standalone.conf` file.

```
# Configuration to enable Basic authentication
authenticationEnabled=true
authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderBasic
To configure brokers to authenticate clients, add the following parameters to the `conf/broker.conf` file. If you use a standalone Pulsar, you need to add these parameters to the `conf/standalone.conf` file.

```conf
# Configuration to enable Basic authentication
authenticationEnabled=true
authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderBasic
basicAuthConf=file:///path/to/.htpasswd
# basicAuthConf=/path/to/.htpasswd
# When use the base64 format, you need to encode the .htpaswd content to bas64
# basicAuthConf=data:;base64,YOUR-BASE64
# basicAuthConf=YOUR-BASE64
# Authentication settings of the broker itself. Used when the broker connects to other brokers, either in same or other clusters
brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationBasic
brokerClientAuthenticationParameters={"userId":"superuser","password":"admin"}
# If this flag is set then the broker authenticates the original Auth data
# else it just accepts the originalPrincipal and authorizes it (if required).
authenticateOriginalAuthData=true
```

# Authentication settings of the broker itself. Used when the broker connects to other brokers, either in same or other clusters
brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationBasic
brokerClientAuthenticationParameters={"userId":"superuser","password":"admin"}
:::note

# If this flag is set then the broker authenticates the original Auth data
# else it just accepts the originalPrincipal and authorizes it (if required).
authenticateOriginalAuthData=true
```
You can also set an environment variable named `PULSAR_EXTRA_OPTS` and the value is `-Dpulsar.auth.basic.conf=/path/to/.htpasswd`. Pulsar reads this environment variable to implement HTTP basic authentication.

2. Set an environment variable named `PULSAR_EXTRA_OPTS` and the value is `-Dpulsar.auth.basic.conf=/path/to/.htpasswd`. Pulsar reads this environment variable to implement HTTP basic authentication.
:::

## Enable basic authentication on proxies

To configure proxies to authenticate clients, complete the following steps.

1. Add the following parameters to the `conf/proxy.conf` file:

```
# For clients connecting to the proxy
authenticationEnabled=true
authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderBasic
To configure proxies to authenticate clients, add the following parameters to the `conf/proxy.conf` file.

```conf
# For clients connecting to the proxy
authenticationEnabled=true
authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderBasic
basicAuthConf=file:///path/to/.htpasswd
# basicAuthConf=/path/to/.htpasswd
# When use the base64 format, you need to encode the .htpaswd content to bas64
# basicAuthConf=data:;base64,YOUR-BASE64
# basicAuthConf=YOUR-BASE64
# For the proxy to connect to brokers
brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationBasic
brokerClientAuthenticationParameters={"userId":"superuser","password":"admin"}
# Whether client authorization credentials are forwarded to the broker for re-authorization.
# Authentication must be enabled via authenticationEnabled=true for this to take effect.
forwardAuthorizationCredentials=true
```

# For the proxy to connect to brokers
brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationBasic
brokerClientAuthenticationParameters={"userId":"superuser","password":"admin"}
:::note

# Whether client authorization credentials are forwarded to the broker for re-authorization.
# Authentication must be enabled via authenticationEnabled=true for this to take effect.
forwardAuthorizationCredentials=true
```
You can also set an environment variable named `PULSAR_EXTRA_OPTS` and the value is `-Dpulsar.auth.basic.conf=/path/to/.htpasswd`. Pulsar reads this environment variable to implement HTTP basic authentication.

2. Set an environment variable named `PULSAR_EXTRA_OPTS` and the value is `-Dpulsar.auth.basic.conf=/path/to/.htpasswd`. Pulsar reads this environment variable to implement HTTP basic authentication.
:::

## Configure basic authentication in CLI tools

[Command-line tools](/docs/next/reference-cli-tools), such as [Pulsar-admin](/tools/pulsar-admin/), [Pulsar-perf](/tools/pulsar-perf/) and [Pulsar-client](/tools/pulsar-client/), use the `conf/client.conf` file in your Pulsar installation. To configure basic authentication in Pulsar CLI tools, you need to add the following parameters to the `conf/client.conf` file.

```
```conf
authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationBasic
authParams={"userId":"superuser","password":"admin"}
```
Expand All @@ -124,4 +136,20 @@ The following example shows how to configure basic authentication when using Pul
```

</TabItem>
<TabItem value="C++" label="C++" default>

```c++
#include <pulsar/Client.h>

int main() {
pulsar::ClientConfiguration config;
AuthenticationPtr auth = pulsar::AuthBasic::create("admin", "123456")
config.setAuth(auth);
pulsar::Client client("pulsar://broker.example.com:6650/", config);
return 0;
}
```
</TabItem>
</Tabs>
94 changes: 61 additions & 33 deletions site2/website/versioned_docs/version-2.8.x/security-basic-auth.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,9 @@ Install [`htpasswd`](https://httpd.apache.org/docs/2.4/programs/htpasswd.html) i
## Create your authentication file

:::note

Currently, you can use MD5 (recommended) and CRYPT encryption to authenticate your password.

:::

Create a password file named `.htpasswd` with a user account `superuser/admin`:
Expand All @@ -55,53 +57,63 @@ superuser:$apr1$GBIYZYFZ$MzLcPrvoUky16mLcK6UtX/

## Enable basic authentication on brokers

To configure brokers to authenticate clients, complete the following steps.

1. Add the following parameters to the `conf/broker.conf` file. If you use a standalone Pulsar, you need to add these parameters to the `conf/standalone.conf` file.

```
# Configuration to enable Basic authentication
authenticationEnabled=true
authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderBasic
To configure brokers to authenticate clients, add the following parameters to the `conf/broker.conf` file. If you use a standalone Pulsar, you need to add these parameters to the `conf/standalone.conf` file.

```conf
# Configuration to enable Basic authentication
authenticationEnabled=true
authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderBasic
basicAuthConf=file:///path/to/.htpasswd
# basicAuthConf=/path/to/.htpasswd
# When use the base64 format, you need to encode the .htpaswd content to bas64
# basicAuthConf=data:;base64,YOUR-BASE64
# basicAuthConf=YOUR-BASE64
# Authentication settings of the broker itself. Used when the broker connects to other brokers, either in same or other clusters
brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationBasic
brokerClientAuthenticationParameters={"userId":"superuser","password":"admin"}
# If this flag is set then the broker authenticates the original Auth data
# else it just accepts the originalPrincipal and authorizes it (if required).
authenticateOriginalAuthData=true
```

# Authentication settings of the broker itself. Used when the broker connects to other brokers, either in same or other clusters
brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationBasic
brokerClientAuthenticationParameters={"userId":"superuser","password":"admin"}
:::note

# If this flag is set then the broker authenticates the original Auth data
# else it just accepts the originalPrincipal and authorizes it (if required).
authenticateOriginalAuthData=true
```
You can also set an environment variable named `PULSAR_EXTRA_OPTS` and the value is `-Dpulsar.auth.basic.conf=/path/to/.htpasswd`. Pulsar reads this environment variable to implement HTTP basic authentication.

2. Set an environment variable named `PULSAR_EXTRA_OPTS` and the value is `-Dpulsar.auth.basic.conf=/path/to/.htpasswd`. Pulsar reads this environment variable to implement HTTP basic authentication.
:::

## Enable basic authentication on proxies

To configure proxies to authenticate clients, complete the following steps.

1. Add the following parameters to the `conf/proxy.conf` file:

```
# For clients connecting to the proxy
authenticationEnabled=true
authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderBasic
To configure proxies to authenticate clients, add the following parameters to the `conf/proxy.conf` file.

```conf
# For clients connecting to the proxy
authenticationEnabled=true
authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderBasic
basicAuthConf=file:///path/to/.htpasswd
# basicAuthConf=/path/to/.htpasswd
# When use the base64 format, you need to encode the .htpaswd content to bas64
# basicAuthConf=data:;base64,YOUR-BASE64
# basicAuthConf=YOUR-BASE64
# For the proxy to connect to brokers
brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationBasic
brokerClientAuthenticationParameters={"userId":"superuser","password":"admin"}
# Whether client authorization credentials are forwarded to the broker for re-authorization.
# Authentication must be enabled via authenticationEnabled=true for this to take effect.
forwardAuthorizationCredentials=true
```

# For the proxy to connect to brokers
brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationBasic
brokerClientAuthenticationParameters={"userId":"superuser","password":"admin"}
:::note

# Whether client authorization credentials are forwarded to the broker for re-authorization.
# Authentication must be enabled via authenticationEnabled=true for this to take effect.
forwardAuthorizationCredentials=true
```
You can also set an environment variable named `PULSAR_EXTRA_OPTS` and the value is `-Dpulsar.auth.basic.conf=/path/to/.htpasswd`. Pulsar reads this environment variable to implement HTTP basic authentication.

2. Set an environment variable named `PULSAR_EXTRA_OPTS` and the value is `-Dpulsar.auth.basic.conf=/path/to/.htpasswd`. Pulsar reads this environment variable to implement HTTP basic authentication.
:::

## Configure basic authentication in CLI tools

[Command-line tools](/docs/next/reference-cli-tools), such as [Pulsar-admin](/tools/pulsar-admin/), [Pulsar-perf](/tools/pulsar-perf/) and [Pulsar-client](/tools/pulsar-client/), use the `conf/client.conf` file in your Pulsar installation. To configure basic authentication in Pulsar CLI tools, you need to add the following parameters to the `conf/client.conf` file.

```
```conf
authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationBasic
authParams={"userId":"superuser","password":"admin"}
```
Expand All @@ -124,4 +136,20 @@ The following example shows how to configure basic authentication when using Pul
```

</TabItem>
<TabItem value="C++" label="C++" default>

```c++
#include <pulsar/Client.h>

int main() {
pulsar::ClientConfiguration config;
AuthenticationPtr auth = pulsar::AuthBasic::create("admin", "123456")
config.setAuth(auth);
pulsar::Client client("pulsar://broker.example.com:6650/", config);
return 0;
}
```
</TabItem>
</Tabs>
30 changes: 30 additions & 0 deletions site2/website/versioned_docs/version-2.9.x/io-debezium-source.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ The configuration of Debezium source connector has the following properties.
| `database.history.pulsar.service.url` | true | null | Pulsar cluster service URL for history topic. |
| `offset.storage.topic` | true | null | Record the last committed offsets that the connector successfully completes. |
| `json-with-envelope` | false | false | Present the message only consist of payload.
| `database.history.pulsar.reader.config` | false | null | The configs of the reader for the database schema history topic, in the form of a JSON string with key-value pairs. <br />**Note:** This property is only available in 2.9.4 and later versions. |
| `offset.storage.reader.config` | false | null | The configs of the reader for the kafka connector offsets topic, in the form of a JSON string with key-value pairs. <br />**Note:** This property is only available in 2.9.4 and later versions.|

### Converter Options

Expand All @@ -55,7 +57,35 @@ Schema.AUTO_CONSUME(), KeyValueEncodingType.SEPARATED)`, and the message consist
| `mongodb.password` | true | null | Password to be used when connecting to MongoDB. This is required only when MongoDB is configured to use authentication. |
| `mongodb.task.id` | true | null | The taskId of the MongoDB connector that attempts to use a separate task for each replica set. |

### Customize the Reader config for the metadata topics

:::note

The customization is only available in 2.9.4 and later versions.

:::

The Debezium Connector exposes `database.history.pulsar.reader.config` and `offset.storage.reader.config` to configure the reader of database schema history topic and the Kafka connector offsets topic. For example, it can be used to configure the subscription name and other reader configurations. You can find the available configurations at [ReaderConfigurationData](https://github.com/apache/pulsar/blob/master/pulsar-client/src/main/java/org/apache/pulsar/client/impl/conf/ReaderConfigurationData.java).

For example, to configure the subscription name for both Readers, you can add the following configuration:
* JSON

```json
{
"configs": {
"database.history.pulsar.reader.config": "{\"subscriptionName\":\"history-reader\"}",
"offset.storage.reader.config": "{\"subscriptionName\":\"offset-reader\"}",
}
}
```

* YAML

```yaml
configs:
database.history.pulsar.reader.config: "{\"subscriptionName\":\"history-reader\"}"
offset.storage.reader.config: "{\"subscriptionName\":\"offset-reader\"}"
```

## Example of MySQL

Expand Down

0 comments on commit d4cb371

Please sign in to comment.