You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
KAFKA-19193 Support rack-aware partitioning for Kafka producer
According to KIP-1123, this commit adds the support for rack-aware partitioning to `BuiltInPartitioner`. It comes with two new configs for the producer: `partitioner.rack.aware` and `client.rack`, which allows enabling the new behavior.
Apart from the added unit tests, the desired behavior was validated by `kafka-producer-perf-test.sh` with an existing and a non-existing rack against a 4 node cluster with two racks and 12-partition topic:
```shell
./kafka_2.13-4.1.0-SNAPSHOT/bin/kafka-producer-perf-test.sh \
--topic test-topic --num-records 100000 --throughput -1 --record-size 1 \
--producer-props bootstrap.servers=127.0.0.1:9092 \
client.rack=rack0 partitioner.rack.aware=true
```
privatestaticfinalStringPARTITIONER_RACK_AWARE_DOC = "Controls whether the default partitioner is rack-aware. This has no effect when a custom partitioner is used.";
125
+
122
126
/** <code>acks</code> */
123
127
publicstaticfinalStringACKS_CONFIG = "acks";
124
128
privatestaticfinalStringACKS_DOC = "The number of acknowledgments the producer requires the leader to have received before considering a request complete. This controls the "
@@ -175,6 +179,12 @@ public class ProducerConfig extends AbstractConfig {
@@ -314,7 +324,9 @@ public class ProducerConfig extends AbstractConfig {
314
324
"This strategy send records to a partition until at least " + BATCH_SIZE_CONFIG + " bytes is produced to the partition. It works with the strategy:" +
315
325
"<ol>" +
316
326
"<li>If no partition is specified but a key is present, choose a partition based on a hash of the key.</li>" +
317
-
"<li>If no partition or key is present, choose the sticky partition that changes when at least " + BATCH_SIZE_CONFIG + " bytes are produced to the partition.</li>" +
327
+
"<li>If no partition or key is present, choose the sticky partition that changes when at least <code>" + BATCH_SIZE_CONFIG + "</code> bytes are produced to the partition.</li>" +
328
+
"<li>If <code>" + CLIENT_RACK_CONFIG + "</code> is specified and <code>" + PARTITIONER_RACK_AWARE_CONFIG + "=true</code>, the sticky partition is chosen from partitions " +
329
+
"with the leader broker in the same rack, if at least one is available. If none are available, it falls back on selecting from all available partitions.</li>" +
318
330
"</ol>" +
319
331
"</li>" +
320
332
"<li><code>org.apache.kafka.clients.producer.RoundRobinPartitioner</code>: A partitioning strategy where " +
@@ -394,9 +406,11 @@ public class ProducerConfig extends AbstractConfig {
0 commit comments