Convert testing suite to TestContainers.

[j7nw4r]

The current test suite seems to fail due to VMWare/Broadcom's deprecation of Bitnami (shakes fist).

To get testing back on track, I suggest that we migrate the testing to use Kafka Testcontainers.

Essentially, each test compilation unit (each test file) would run it's own docker Kafka container. There are even multiple versions available (Apache vs Confluent, Kraft vs Zookeeper).

The first pass would would be implementation using Apache Kraft Kafka. Eventually I think we could create a test macro would would enumerate the tests over all of the necessary Kafka distributions.

This change would bring in a dependency on local Docker/Podman being available for testing, but I don't think that is a big deal given that the current test suite depends on external Kafka.

Thoughts? If we're in agreement, I can take this one.

https://docs.rs/testcontainers/latest/testcontainers/
https://docs.rs/testcontainers-modules/latest/testcontainers_modules/

@fede1024

[fede1024]

Yes, this would be great! The test suite for this library is overdue for a revamp. In hindsight moving to the bitnami images was not a great idea. No objections on my side, thank you for working on this!

…

On Thu, 2 Oct 2025, 19:42 Johnathan W, ***@***.***> wrote: *j7nw4r* created an issue (fede1024/rust-rdkafka#797) <#797> The current test suite seems to fail due to VMWare/Broadcom's deprecation of Binami (*shakes fist*). To get testing back on track, I suggest that we migrate the testing to use Kafka Testcontainers <https://testcontainers.com/modules/kafka/?language=rust>. Essentially, each test compilation unit (each test file) would run it's own docker Kafka container. There are even multiple versions available (Apache vs Confluent, Kraft vs Zookeeper). The first pass would would be implementation using Apache Kraft Kafka. Eventually I think we could create a test macro would would enumerate the tests over all of the necessary Kafka distributions. This change would bring in a dependency on local Docker/Podman being available for testing, but I don't think that is a big deal given that the current test suite depends on external Kafka. Thoughts? If we're in agreement, I can take this one. https://docs.rs/testcontainers/latest/testcontainers/ https://docs.rs/testcontainers-modules/0.13.0/testcontainers_modules/ @fede1024 <https://github.com/fede1024> — Reply to this email directly, view it on GitHub <#797>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALY3T3Z3GNGLTYDKC3MZXD3VVWZ3AVCNFSM6AAAAACIEKE7NOVHI2DSMVQWIX3LMV43ASLTON2WKOZTGQ3TQNJTGY4DOMI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[j7nw4r]

Work can be tracked here: #798

[j7nw4r]

@fede1024 Just for clarity before I continue, this is a what a basic produce test looks like: https://github.com/fede1024/rust-rdkafka/pull/798/files#diff-7f604488cb54e565cbbffe2475cac35b642abcd7bc9b2a2bcd26130c3219dcd5.

This should give an idea of how test containers are used.

[fede1024]

That looks great! Thank you for taking the time to implement tests properly instead of just updating the docker image. Hopefully once the initial setup code is implemented (and it looks like it is), porting more tests won't require too much work.