[bitnami/kafka] Kafka JMX Exporter , Port already in use: 5555

[bharathreddy901]

Name and Version

bitnami/kafka

What steps will reproduce the bug?

Hi Team,

We are using bitnami Kafka helm charts. We have enabled the JMX Exporter.

When we ran the Kafka console commands, I am getting below error(java.rmi.server.ExportException: Port already in use: 5555)
For WorkAround,I am running below commands.

export JMX_PORT=5557
unset JMX_PORT

Is there any permanent solution to fix this.

I have no name!@kafka-0:/$ kafka-topics.sh --list --bootstrap-server kafka-0.kafka-headless.kafka.svc.cluster.local:9092
Error: Exception thrown by the agent : java.rmi.server.ExportException: Port already in use: 5555; nested exception is:
        java.net.BindException: Address already in use (Bind failed)
jdk.internal.agent.AgentConfigurationError: java.rmi.server.ExportException: Port already in use: 5555; nested exception is:
        at jdk.management.agent/sun.management.jmxremote.ConnectorBootstrap.startRemoteConnectorServer(ConnectorBootstrap.java:491)
        at jdk.management.agent/jdk.internal.agent.Agent.startAgent(Agent.java:447)
        at jdk.management.agent/jdk.internal.agent.Agent.startAgent(Agent.java:599)
Caused by: java.rmi.server.ExportException: Port already in use: 5555; nested exception is:

Thanks.

Are you using any custom parameters or values?

No response

What is the expected behavior?

No response

What do you see instead?

java.net.BindException: Address already in use (Bind failed)

Additional information

No response

[bharathreddy901]

Hi @fmulero @javsalgar : Please help me on above issue.

[fmulero]

Hi @bharathreddy901

When JMX exporter is enabled, JMX port in kafka container is set to 5555 and jmx-exporter sidecar use it to collect the metrics and expose them in port 5556. Kafka commands use the same env var (JMX_PORT), so they will try to open the port configured for the server.

As workaround you can run your commands in a separate client:

$ kubectl run kafka-client --restart='Never' -it --image docker.io/bitnami/kafka:3.3.1-debian-11-r1 --namespace default --command -- bash
I have no name!@kafka-client:/$ kafka-topics.sh --list --bootstrap-server kafka.default.svc.cluster.local:9092

[bharathreddy901]

Thank you @fmulero, for your inputs. I am following above workaround. Once it is fixed, Please let me know. Is there any grafana dashboards for Kafka? can you please share it for troubleshooting purpose.

[fmulero]

There is no real issue to fix here, clients should run outside of the server pods.

If you like to contribute using a different JMX listener configuration for the kafka container The Bitnami team will be happy to review it and provide feedback. Here you can find the contributing guidelines.

[bharathreddy901]

Thank you @fmulero for your inputs.

[dellnoantechnp]

if you set metrics.jmx.enabled value to true, then usage like this:

$  JMX_PORT=0   kafka-console-consumer.sh  --bootstrap-server localhost:9092 --topic test