KafkaServerStartable removed in Kafka 2.8

[aka-peter]

I stumbled upon this when testing a quarkus based project using Kafka 2.8.

apache/kafka#9883

[dhenneke]

I tried to fix this but for some reason certain tests kept failing. My idea was to use reflection to switch to another class for 2.8.0:

-  private KafkaServerStartable broker;
+  private KafkaServer broker;

// ...

-   broker = new KafkaServerStartable(brokerConfig);
+  try {
+    // use KafkaServerStartable for kafka_2.12 <2.8.0
+    Class<?> cl = Class.forName("kafka.server.KafkaServerStartable");
+    Constructor<?> constructor = cl.getConstructor(KafkaConfig.class);
+    broker = (KafkaServer) constructor.newInstance(brokerConfig);
+  } catch (ClassNotFoundException e) {
+    // use KafkaServer for kafka_2.12 >=2.8.0
+    Class<?> cl = Class.forName("kafka.server.KafkaServer");
+    Constructor<?> constructor = cl.getConstructor(KafkaConfig.class, Time.class, Option.class, boolean.class);
+    broker = (KafkaServer) constructor.newInstance(brokerConfig, Time.SYSTEM, Option.<String>empty(), false);
+  }

We could also use KafkaServer for <2.8.0 since we don't use any feature of the wrapping KafkaServerStartable, but the constructor interface of this class also changed.

Not sure if this is a good approach. Someone else will probably have a better idea on how to fix this. Especially regarding future-proofness, we probably don't want to run into chained try-catch statements for each future kafka version...

[dlipofsky]

I also ran into this. Trying to use kafka-junit with kafka 2.8.0 I got
ClassNotFoundException: kafka.server.KafkaServerStartable
and with kafka 2.7.1 I got
ClassNotFoundException: scala.math.Ordering$$anon$5

To be fair, the README for this project says it supports through 2.6.x. But is there any intent to go beyond 2.6.x?

[mikaello]

See also discussion in issue #49 , it seems at least not an intention to stay at 2.6.x.

[dlipofsky]

For those looking for a solution now, I was able to switch to net.mguenther.kafka:kafka-junit:2.8.0 fairly painlessly. Luckily I only had 4 simple unit tests that needed broker interaction.

[Crim]

Sorry for the delay, I've spent some time today making some organizational changes to the library (switch from travisCI to Github actions) and officially added support for Kafka 2.7.x.

I'll try to dig into how best to support 2.8.x and above, hopefully in a way that maintains backwards compatibility.

[Crim]

I released v3.2.3 to maven central. It should support 2.8.x and be fully compatible with previous versions. Let me know if you run into any troubles.

I ended up going more or less with your approach @dhenneke as it seemed like the easiest path forward. If this needs to be modified again the future we could optimize it by looking at the packaged AppProperties in Kafka to parse out the version and go straight for the correct constructor.

Thanks!

[dhenneke]

@Crim Great, thank you! I updated it in our library and our tests are green.

[Crim]

awesome! Going to close out this issue.

Thanks!