Use interface versions for Producer and Consumer

[empperi]

Currently code uses directly org.apache.kafka.clients.producer.KafkaProducer and org.apache.kafka.clients.consumer.KafkaConsumer. Code should use org.apache.kafka.clients.producer.Producer and org.apache.kafka.clients.consumer.Consumer interfaces instead.

Why this matters? Because Kafka Java client library provides both MockConsumer and MockProducer out of the box which implement those interfaces. This allows one to use those mock implementations in place of the actual Kafka* versions when doing unit testing. Now however since code uses the Kafka* versions directly with type hints in function signature one gets a java.lang.ClassCastException when calling those functions with mock implementations.

One can go around this via proxy but that requires manual setup and creating your own mock implementations. Since Kafka Client library provides these out of the box they should be preferred.

[source-c]

You're right - the library still don't cover native Java UT. It is designed for solving real production needs where, strictly speaking, it successfully used in pure Clojure environment for years.
But anyway - thanks a lot for you interest and suggestion. I will pay special attention to this issue asap.

[empperi]

I have no doubt it isn't used in production. The thing is, I want to test my implementation. Since library is using the implementations of mentioned interfaces I was forced to implement proxy versions. And I am still implementing those with small gotchas here and there.

Would be much more simple if one could just use the mock versions coming with Kafka Java client library, right? :) Even better if clj-kafka-x did provide some nice Clojure interop into using those!

[empperi]

This actually just became a bit bigger issue than I originally thought.

As it is now, it is practically impossible to mock KafkaProducer via proxy in such a way that it does not require a real Kafka to operate. This is due to package private methods and constructors within KafkaProducer.

And because of that, it is practically impossible to write tests with clj-kafka-x in such way that there is no real Kafka under the hood. This is very much sub-optimal in many ways. Naturally there should be tests that use real Kafka too, but it is a bit much to ask for for whole applications and their testing.

All of this could be fixed very simply. I might create a PR for this.