Skip to content

Commit

Permalink
KAFKA-9563: Fix Kafka Connect documentation around consumer and produ…
Browse files Browse the repository at this point in the history
…cer overrides (#8124)

Kafka Connect main doc required a fix to distinguish between worker level producer and consumer overrides and per-connector level producer and consumer overrides, after the latter were introduced with KIP-458.  

* [KAFKA-9563] Fix Kafka connect consumer and producer override documentation

Co-authored-by: Konstantine Karantasis <konstantine@confluent.io>

Reviewers: Konstantine Karantasis <konstantine@confluent.io>
  • Loading branch information
Hossein Torabi authored and kkonstantine committed Mar 23, 2020
1 parent f9d0ec3 commit e2c58be
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion docs/connect.html
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,9 @@ <h4><a id="connect_running" href="#connect_running">Running Kafka Connect</a></h
<li><code>offset.storage.file.filename</code> - File to store offset data in</li>
</ul>
<p>The parameters that are configured here are intended for producers and consumers used by Kafka Connect to access the configuration, offset and status topics. For configuration of Kafka source and Kafka sink tasks, the same parameters can be used but need to be prefixed with <code>consumer.</code> and <code>producer.</code> respectively. The only parameter that is inherited from the worker configuration is <code>bootstrap.servers</code>, which in most cases will be sufficient, since the same cluster is often used for all purposes. A notable exception is a secured cluster, which requires extra parameters to allow connections. These parameters will need to be set up to three times in the worker configuration, once for management access, once for Kafka sinks and once for Kafka sources.</p>
<p>The parameters that are configured here are intended for producers and consumers used by Kafka Connect to access the configuration, offset and status topics. For configuration of the producers used by Kafka source tasks and the consumers used by Kafka sink tasks, the same parameters can be used but need to be prefixed with <code>producer.</code> and <code>consumer.</code> respectively. The only Kafka client parameter that is inherited without a prefix from the worker configuration is <code>bootstrap.servers</code>, which in most cases will be sufficient, since the same cluster is often used for all purposes. A notable exception is a secured cluster, which requires extra parameters to allow connections. These parameters will need to be set up to three times in the worker configuration, once for management access, once for Kafka sources and once for Kafka sinks.</p>
<p>Starting with 2.3.0, client configuration overrides can be configured individually per connector by using the prefixes <code>producer.override.</code> and <code>consumer.override.</code> for Kafka sources or Kafka sinks respectively. These overrides are included with the rest of the connector's configuration properties.</p>

<p>The remaining parameters are connector configuration files. You may include as many as you want, but all will execute within the same process (on different threads).</p>

Expand Down

0 comments on commit e2c58be

Please sign in to comment.