Simplify Kafka container by deferring the Kafka command #1458

bsideup · 2019-05-10T18:42:10Z

It is possible to remove the Socat & networks from KafkaContainer.

The algorithm:

Start container with command sleep infinity
After it started (but before the waits):
1. If external ZK is not configured, start an internal one with docker exec ...
2. start Kafka with docker exec ...

modules/kafka/src/main/java/org/testcontainers/containers/KafkaContainer.java

kiview · 2019-05-11T21:32:01Z

modules/kafka/src/main/java/org/testcontainers/containers/KafkaContainer.java

+    @NonNull
+    public synchronized Network getNetwork() {
+        if (super.getNetwork() == null) {
+            // Backward compatibility


Do we really want to add this for backward compatibility? IMO the network was an implementation detail before, not really part of the public API of the class.

Is the timing risky here? As in:
(a) could the container come up automatically on the bridge network, but then return a completely different network to user code when this method is called?
(b) Is it possible that this code will be invoked after the container is started, meaning that withNetwork is not going to do the right thing...

I've reverted the change, let's tackle it later (added a TODO)

modules/kafka/src/main/java/org/testcontainers/containers/KafkaContainer.java

…iners#1312) Bumps [assertj-core](https://github.com/joel-costigliola/assertj-core) from 3.12.1 to 3.12.2. <details> <summary>Commits</summary> - [`5dd7fac`](assertj/assertj@5dd7fac) [maven-release-plugin] prepare release assertj-core-3.12.2 - [`d427863`](assertj/assertj@d427863) Update Guava test dependency to version 27.1-jre. - [`cb5b285`](assertj/assertj@cb5b285) anySatisfy (Maps): do not continue evaluating elements once we found a match ... - [`8d83c41`](assertj/assertj@8d83c41) Ignore static methods when finding property accessors. Fixes [testcontainers#1458](https://github-redirect.dependabot.com/joel-costigliola/assertj-core/issues/1458) - [`b600f5b`](assertj/assertj@b600f5b) Use URI#create (which does not throw a checked Exception) in tests. - [`91af5c9`](assertj/assertj@91af5c9) [maven-release-plugin] prepare for next development iteration - See full diff in [compare view](assertj/assertj@assertj-core-3.12.1...assertj-core-3.12.2) </details> <br /> [![Dependabot compatibility score](https://api.dependabot.com/badges/compatibility_score?dependency-name=org.assertj:assertj-core&package-manager=gradle&previous-version=3.12.1&new-version=3.12.2)](https://dependabot.com/compatibility-score.html?dependency-name=org.assertj:assertj-core&package-manager=gradle&previous-version=3.12.1&new-version=3.12.2) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) If all status checks pass Dependabot will automatically merge this pull request. [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot ignore this [patch|minor|major] version` will close this PR and stop Dependabot creating any more for this minor/major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot badge me` will comment on this PR with code to add a "Dependabot enabled" badge to your readme Additionally, you can set the following in the `.dependabot/config.yml` file in this repo: - Update frequency (including time of day and day of week) - Automerge options (never/patch/minor, and dev/runtime dependencies) - Pull request limits (per update run and/or open at any time) - Out-of-range updates (receive only lockfile updates, if desired) - Security updates (receive only security updates, if desired) Finally, you can contact us by mentioning @dependabot. </details>

[//]: # (dependabot-start) ⚠️ **Dependabot is rebasing this PR** ⚠️ If you make any changes to it yourself then they will take precedence over the rebase. --- [//]: # (dependabot-end) Bumps [assertj-core](https://github.com/joel-costigliola/assertj-core) from 3.12.1 to 3.12.2. <details> <summary>Commits</summary> - [`5dd7fac`](assertj/assertj@5dd7fac) [maven-release-plugin] prepare release assertj-core-3.12.2 - [`d427863`](assertj/assertj@d427863) Update Guava test dependency to version 27.1-jre. - [`cb5b285`](assertj/assertj@cb5b285) anySatisfy (Maps): do not continue evaluating elements once we found a match ... - [`8d83c41`](assertj/assertj@8d83c41) Ignore static methods when finding property accessors. Fixes [testcontainers#1458](https://github-redirect.dependabot.com/joel-costigliola/assertj-core/issues/1458) - [`b600f5b`](assertj/assertj@b600f5b) Use URI#create (which does not throw a checked Exception) in tests. - [`91af5c9`](assertj/assertj@91af5c9) [maven-release-plugin] prepare for next development iteration - See full diff in [compare view](assertj/assertj@assertj-core-3.12.1...assertj-core-3.12.2) </details> <br /> [![Dependabot compatibility score](https://api.dependabot.com/badges/compatibility_score?dependency-name=org.assertj:assertj-core&package-manager=gradle&previous-version=3.12.1&new-version=3.12.2)](https://dependabot.com/compatibility-score.html?dependency-name=org.assertj:assertj-core&package-manager=gradle&previous-version=3.12.1&new-version=3.12.2) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) If all status checks pass Dependabot will automatically merge this pull request. [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot ignore this [patch|minor|major] version` will close this PR and stop Dependabot creating any more for this minor/major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot badge me` will comment on this PR with code to add a "Dependabot enabled" badge to your readme Additionally, you can set the following in the `.dependabot/config.yml` file in this repo: - Update frequency (including time of day and day of week) - Automerge options (never/patch/minor, and dev/runtime dependencies) - Pull request limits (per update run and/or open at any time) - Out-of-range updates (receive only lockfile updates, if desired) - Security updates (receive only security updates, if desired) Finally, you can contact us by mentioning @dependabot. </details>

rnorth · 2019-05-15T08:24:14Z

modules/kafka/src/main/java/org/testcontainers/containers/KafkaContainer.java

@@ -54,40 +56,67 @@ public KafkaContainer withExternalZookeeper(String connectString) {
    }

    public String getBootstrapServers() {
-        return String.format("PLAINTEXT://%s:%s", proxy.getContainerIpAddress(), proxy.getFirstMappedPort());
+        if (port == -1) {
+            throw new IllegalStateException("You should start Kafka container first");


Nice - this will fix #1450

Yeah, this is cute :)

rnorth

Looks good on the first pass!

rnorth · 2019-05-15T08:27:55Z

modules/kafka/src/main/java/org/testcontainers/containers/KafkaContainer.java

+    @NonNull
+    public synchronized Network getNetwork() {
+        if (super.getNetwork() == null) {
+            // Backward compatibility


Is the timing risky here? As in:
(a) could the container come up automatically on the bridge network, but then return a completely different network to user code when this method is called?
(b) Is it possible that this code will be invoked after the container is started, meaning that withNetwork is not going to do the right thing...

modules/kafka/src/main/java/org/testcontainers/containers/KafkaContainer.java

Simplify Kafka container by deferring the Kafka command

1176da6

bsideup added type/feature modules/kafka labels May 10, 2019

bsideup added this to the next milestone May 10, 2019

bsideup requested a review from a team May 10, 2019 18:42

bsideup requested review from kiview and rnorth as code owners May 10, 2019 18:42

kiview requested changes May 11, 2019

View reviewed changes

rnorth reviewed May 15, 2019

View reviewed changes

bsideup and others added 2 commits May 16, 2019 13:19

extract startZookeeper method, add PORT_NOT_ASSIGNED constant

ae23b88

Merge branch 'master' into simplify_kafka

a807314

bsideup mentioned this pull request Jun 1, 2019

Add e2e review sysco-middleware/kafka-testing#4

Merged

rnorth modified the milestone: next Jul 7, 2019

keep network&alias for backward compatibility

f248e79

bsideup requested a review from kiview July 16, 2019 11:55

bsideup added 2 commits July 16, 2019 21:09

Merge branch 'master' into simplify_kafka

0e95b0b

Merge branch 'master' into simplify_kafka

c526b3d

kiview approved these changes Jul 17, 2019

View reviewed changes

bsideup merged commit b840912 into master Jul 17, 2019

delete-merged-branch bot deleted the simplify_kafka branch July 17, 2019 13:19

bsideup mentioned this pull request Dec 18, 2019

Create a "pattern" for NAT-unfriendly systems (Kafka, Couchbase, etc) #1940

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Simplify Kafka container by deferring the Kafka command #1458

Simplify Kafka container by deferring the Kafka command #1458

bsideup commented May 10, 2019 •

edited

Loading

kiview May 11, 2019

rnorth May 15, 2019

bsideup Jul 16, 2019

rnorth May 15, 2019

kiview Jul 17, 2019

rnorth left a comment

rnorth May 15, 2019

Simplify Kafka container by deferring the Kafka command #1458

Simplify Kafka container by deferring the Kafka command #1458

Conversation

bsideup commented May 10, 2019 • edited Loading

kiview May 11, 2019

Choose a reason for hiding this comment

rnorth May 15, 2019

Choose a reason for hiding this comment

bsideup Jul 16, 2019

Choose a reason for hiding this comment

rnorth May 15, 2019

Choose a reason for hiding this comment

kiview Jul 17, 2019

Choose a reason for hiding this comment

rnorth left a comment

Choose a reason for hiding this comment

rnorth May 15, 2019

Choose a reason for hiding this comment

bsideup commented May 10, 2019 •

edited

Loading