kafka: extend client.id parsing per producer/consumer #426
Conversation
jeqo
force-pushed
the
kafka-client-id-per-prod-cons
branch
from
0a00002 to
874db8a
Compare
|
|
||
| When running workers, pass the `zone.id`: | ||
|
|
||
| ```bash |
| /opt/benchmark/bin/benchmark-worker | ||
| ``` | ||
|
|
||
| Then pass the `client.id` template: |
There was a problem hiding this comment.
Why not use something like envsubst
| private Properties topicProperties; | ||
| private Properties producerProperties; | ||
| private Properties consumerProperties; | ||
| // Visible for testing |
There was a problem hiding this comment.
I'd use a Hashmap/table + enum for the three types
| } | ||
|
|
||
| producerProperties = new Properties(); | ||
| commonProperties.forEach((key, value) -> producerProperties.put(key, value)); |
| producerProperties.load(new StringReader(config.producerConfig)); | ||
|
|
||
| if (producerProperties.containsKey(KAFKA_CLIENT_ID)) { | ||
| producerProperties.put( |
There was a problem hiding this comment.
Isn't there a putIfAbsent inversion?
|
|
||
| try (KafkaBenchmarkDriver driver = new KafkaBenchmarkDriver()) { | ||
| // When initializing kafka driver | ||
| Files.write(configPath, KafkaBenchmarkDriver.mapper.writeValueAsBytes(config)); |
There was a problem hiding this comment.
Use a ByteArray Stream rather than a file
|
|
||
| // Then | ||
| if (producerClientId != null) { | ||
| assertThat(driver.producerProperties).containsEntry("client.id", producerClientId); |
There was a problem hiding this comment.
Assert against CommonConfig constants not magic string
| driver.initialize(configPath.toFile(), null); | ||
|
|
||
| // Then | ||
| if (producerClientId != null) { |
There was a problem hiding this comment.
Nit: Optional types are better than nullable Objects
| } else { | ||
| assertThat(driver.producerProperties).doesNotContainKey("client.id"); | ||
| } | ||
| if (consumerClientId != null) { |
There was a problem hiding this comment.
You should assert nullability based on parameterized test order , otherwise this is just lazy testing
| assertThat(driver.producerProperties).doesNotContainKey("client.id"); | ||
| } | ||
| if (consumerClientId != null) { | ||
| assertThat(driver.consumerProperties).containsEntry("client.id", consumerClientId); |
There was a problem hiding this comment.
I assume containsEntry does the key check short circuit?
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 5 days if no further activity occurs. Thank you for your contributions. |
2 participants
For further flexibility, it may be useful to differentiate client ids when passing a zone id.
Currently, if a zone id variable is needed, it needs to be in common configurations, forcing both producer and consumer to have the same values (with different zones).
This proposal extends the same parsing to client ids from producer and consumers; adds some unit tests; and documents this feature.