Incorrect Devservices config generated for Keycloak with Testcontainers Cloud

[wjglerum]

Describe the bug

When using Devservices for Keycloak with Testcontainers Cloud an incorrect config is generated. The detected port mapping is correct, however the injected urls contain the wrong port.

The port 65299 is detected in the port mappings, however the config gets port 65224 injected and nothing is listening on this port.

This does work correctly for data services, like PostgreSQL and Kafka, see below.

Expected behavior

It does work correctly when using Docker Desktop for Mac instead.

Output of $ sudo lsof -i -n -P | grep TCP

com.docke 90705       wjglerum  134u  IPv6 0x3a899d7578da3c87      0t0    TCP *:49887 (LISTEN)
com.docke 90705       wjglerum  135u  IPv6 0x3a899d7578da3587      0t0    TCP 127.0.0.1:49887->127.0.0.1:49888 (ESTABLISHED)
com.docke 90705       wjglerum  136u  IPv6 0x3a899d757a640687      0t0    TCP *:49889 (LISTEN)
com.docke 90705       wjglerum  137u  IPv6 0x3a899d757a643e87      0t0    TCP 127.0.0.1:49889->127.0.0.1:49978 (ESTABLISHED)
com.docke 90705       wjglerum  138u  IPv6 0x3a899d757a63ea87      0t0    TCP [::1]:49889->[::1]:50099 (ESTABLISHED)
com.docke 90705       wjglerum  139u  IPv6 0x3a899d7578d7a887      0t0    TCP [::1]:49889->[::1]:50100 (ESTABLISHED)
com.docke 90705       wjglerum  140u  IPv6 0x3a899d7578d7e087      0t0    TCP [::1]:49889->[::1]:50101 (ESTABLISHED)
com.docke 90705       wjglerum  141u  IPv6 0x3a899d757a641b87      0t0    TCP [::1]:49889->[::1]:50102 (ESTABLISHED)
com.docke 90705       wjglerum  142u  IPv6 0x3a899d757a643787      0t0    TCP [::1]:49889->[::1]:50103 (ESTABLISHED)
com.docke 90705       wjglerum  143u  IPv6 0x3a899d757a643087      0t0    TCP [::1]:49889->[::1]:50104 (ESTABLISHED)

Actual behavior

The app fails with the following exception:

java.net.ConnectException: Connection refused
	at java.base/sun.nio.ch.Net.pollConnect(Native Method)
	at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672)
	at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:946)
	at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:337)
	at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:710)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)
Resulted in: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: /127.0.0.1:65224
Resulted in: io.quarkus.oidc.OIDCException: OIDC Server is not available

Output of $ sudo lsof -i -n -P | grep TCP

testconta  1914       wjglerum    7u  IPv4 0x3a899d7a59a3f38f      0t0    TCP 127.0.0.1:63011 (LISTEN)
testconta  1914       wjglerum   13u  IPv4 0x3a899d7a5994ee3f      0t0    TCP 172.20.10.3:65117->104.26.3.119:443 (ESTABLISHED)
testconta  1914       wjglerum   15u  IPv4 0x3a899d7a59bb338f      0t0    TCP 172.20.10.3:65145->3.228.161.144:443 (ESTABLISHED)
testconta  1914       wjglerum   16u  IPv4 0x3a899d7a5075938f      0t0    TCP 127.0.0.1:65215 (LISTEN)
testconta  1914       wjglerum   17u  IPv4 0x3a899d7a59bd038f      0t0    TCP 172.20.10.3:65144->34.147.124.196:32289 (ESTABLISHED)
testconta  1914       wjglerum   18u  IPv4 0x3a899d7a5074e38f      0t0    TCP 127.0.0.1:65215->127.0.0.1:65216 (ESTABLISHED)
testconta  1914       wjglerum   19u  IPv4 0x3a899d7a59a4638f      0t0    TCP 127.0.0.1:65299 (LISTEN)
testconta  1914       wjglerum   20u  IPv4 0x3a899d7a59a0038f      0t0    TCP 127.0.0.1:63011->127.0.0.1:65347 (ESTABLISHED)

How to Reproduce?

Use Testcontainers Cloud for Mac instead of Docker Desktop for Mac. And start a project from scratch with the OIDC extension.

Output of uname -a or ver

Darwin Willems-MacBook-Pro.local 21.6.0 Darwin Kernel Version 21.6.0: Wed Aug 10 14:25:27 PDT 2022; root:xnu-8020.141.5~2/RELEASE_X86_64 x86_64

Output of java -version

openjdk version "17.0.3" 2022-04-19 OpenJDK Runtime Environment Temurin-17.0.3+7 (build 17.0.3+7) OpenJDK 64-Bit Server VM Temurin-17.0.3+7 (build 17.0.3+7, mixed mode, sharing)

GraalVM version (if different from Java)

No response

Quarkus version or git rev

2.12.0.Final

Build tool (ie. output of mvnw --version or gradlew --version)

Apache Maven 3.8.6 (84538c9988a25aec085021c365c560670ad80f63) Maven home: /Users/wjglerum/.m2/wrapper/dists/apache-maven-3.8.6-bin/1ks0nkde5v1pk9vtc31i9d0lcd/apache-maven-3.8.6 Java version: 17.0.3, vendor: Eclipse Adoptium, runtime: /Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home Default locale: en_NL, platform encoding: UTF-8 OS name: "mac os x", version: "12.5.1", arch: "x86_64", family: "mac"

Additional information

Testcontainers Cloud version 1.3.3

You can probably ask access to their private beta if needed.

[quarkus-bot]

/cc @geoand, @pedroigor, @sberyozkin, @stuartwdouglas

[wjglerum]

I'm happy to help out and dig further into this btw. Where would I find the parts where the config is generated for the devservices? And does this differ for Keycloak?

[gsmet]

@wjglerum the culprit will probably be in KeycloakDevServicesProcessor. It's where the config is adjusted.

[bsideup]

I believe this is the issue:

quarkus/extensions/oidc/deployment/src/main/java/io/quarkus/oidc/deployment/devservices/keycloak/KeycloakDevServicesProcessor.java

Line 403 in 83e7991

fixedExposedPort = OptionalInt.of(findRandomPort());

quarkus/extensions/oidc/deployment/src/main/java/io/quarkus/oidc/deployment/devservices/keycloak/KeycloakDevServicesProcessor.java

Line 423 in 83e7991

addFixedExposedPort(fixedExposedPort.getAsInt(), KEYCLOAK_PORT);

It assumes that the port will be the same on localhost and Docker host, but that isn't always true.

More on this:
https://bsideup.github.io/posts/testcontainers_fixed_ports/

[wjglerum]

@geoand would you have more insights for this and how to proceed? As I see that you worked on those bits.

[geoand]

I'll try and have a look tomorrow and let you know, but I can't make any promises

[wjglerum]

I just confirmed that this is indeed the problematic bit by making sure it skips that bit of the code by adjusting the code locally and building Quarkus locally. Now it does inject the correct port mapping in the keycloak url while using Testcontainers Cloud.

I believe this is the issue:

quarkus/extensions/oidc/deployment/src/main/java/io/quarkus/oidc/deployment/devservices/keycloak/KeycloakDevServicesProcessor.java

Line 403 in 83e7991

fixedExposedPort = OptionalInt.of(findRandomPort());

[wjglerum]

I'll try and have a look tomorrow and let you know, but I can't make any promises

Thanks! I'll have another look tomorrow too.

[geoand]

So you seem to have something working, no?

[kiview]

I am a little bit out of the loop, but can't Quarkus make use of this existing Keycloak module?
https://github.com/dasniko/testcontainers-keycloak

It is not part of the bundled Testcontianers modules at this point, but @dasniko is an active member of the Testcontainers community, and his implementation does not use fixed ports.

[geoand]

That's a fair question. I personally don't know, but it's definitely worth exploring

[sberyozkin]

@kiview It may be an interesting enhancement idea as its API looks nice but it is hard to know it will work without the fixes done by Georgios, both the one built here and the one created by @dasniko are TestContainer instances so perhaps the simplest option is to address this particular issue here. The local implementation makes it a bit more flexible - ex, 19.0.1 is used locally, we can tune start up commands, and we work with the same version to support both WildFly (as we can expect this one being used for a while) and the newer Quarkus based distros - but it is branched out in that project... I propose to discuss it in Discussions for example...