Connection refused from Python producer to Kafka service in Docker container

[jschulenklopper]

Description

Using https://github.com/confluentinc/cp-docker-images I'm trying to create a Python producer to a Kafka service running as a container in their cp-all-in-one example (with a set of containers for broker, zookeeper, ksql-client and such).

Unfortunately, I get

%3|1536331408.542|FAIL|rdkafka#producer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Connect to ipv4#127.0.0.1:9092 failed: Connection refused

errors on a simple p = Producer({'bootstrap.servers': 'localhost', 'broker.address.family': 'v4'}) call.

How to reproduce

So, I'm using the docker-compose.yml from https://github.com/confluentinc/cp-docker-images/blob/5.0.0-post/examples/cp-all-in-one/docker-compose.yml (using the 5.0.0 branch because master is referring to 5.1.0 images that aren't there, or are not working yet), and do a docker-compose up -d. That gives a collection of running containers.

Then I try to run this simple Python application from another Docker container (just for this Python script):

from confluent_kafka import Producer

print('Trying plain localhost connect')
p1 = Producer({'bootstrap.servers': 'localhost', 'broker.address.family': 'v4'})

print('Trying port 9092 on localhost')
p2 = Producer({'bootstrap.servers': 'localhost:9092', 'broker.address.family': 'v4'})

print('Trying port 29092 on localhost')
p3 = Producer({'bootstrap.servers': 'localhost:29092', 'broker.address.family': 'v4'})

print('Trying to connect to broker (container name)')
p4 = Producer({'bootstrap.servers': 'broker', 'broker.address.family': 'v4'})

but that gives warnings:

%3|1536333862.034|FAIL|rdkafka#producer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Connect to ipv4#127.0.0.1:9092 failed: Connection refused
%3|1536333862.034|ERROR|rdkafka#producer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Connect to ipv4#127.0.0.1:9092 failed: Connection refused
%3|1536333862.034|ERROR|rdkafka#producer-1| [thrd:localhost:9092/bootstrap]: 1/1 brokers are down
%3|1536333862.035|FAIL|rdkafka#producer-2| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Connect to ipv4#127.0.0.1:9092 failed: Connection refused
%3|1536333862.035|ERROR|rdkafka#producer-2| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Connect to ipv4#127.0.0.1:9092 failed: Connection refused
%3|1536333862.035|ERROR|rdkafka#producer-2| [thrd:localhost:9092/bootstrap]: 1/1 brokers are down
%3|1536333862.035|FAIL|rdkafka#producer-3| [thrd:localhost:29092/bootstrap]: localhost:29092/bootstrap: Connect to ipv4#127.0.0.1:29092 failed: Connection refused
%3|1536333862.035|ERROR|rdkafka#producer-3| [thrd:localhost:29092/bootstrap]: localhost:29092/bootstrap: Connect to ipv4#127.0.0.1:29092 failed: Connection refused
%3|1536333862.035|ERROR|rdkafka#producer-3| [thrd:localhost:29092/bootstrap]: 1/1 brokers are down
%3|1536333862.037|FAIL|rdkafka#producer-4| [thrd:broker:9092/bootstrap]: broker:9092/bootstrap: Failed to resolve 'broker:9092': Name or service not known
%3|1536333862.037|ERROR|rdkafka#producer-4| [thrd:broker:9092/bootstrap]: broker:9092/bootstrap: Failed to resolve 'broker:9092': Name or service not known
%3|1536333862.037|ERROR|rdkafka#producer-4| [thrd:broker:9092/bootstrap]: 1/1 brokers are down

BTW, this is my Dockerfile:

FROM python:3
RUN pip install confluent-kafka
ADD producer.py /
CMD [ "python", "./producer.py" ]

Side note: Running the Python script as a shell process, so not from a container, gives another error that the nodename / servname are not provided or known: %3|1536332857.140|FAIL|rdkafka#producer-4| [thrd:broker:9092/bootstrap]: broker:9092/bootstrap: Failed to resolve 'broker:9092': nodename nor servname provided, or not known

Even including that Dockerized Python script into docker-compose.yml by including this as a service:

producer:
  image: python-producer:latest
  hostname: producer
  container_name: producer
  depends_on:
    - zookeeper
    - broker

will not get a working solution.

The problem could be in https://github.com/confluentinc/cp-docker-images/tree/5.0.0-post/examples/cp-all-in-one as well... but I'm not sure. Since I've been able to create topics and post events to Kafka from the CLI in that configuration, I assume the cause of the refused connection is not in the Kafka containers.

Checklist

Please provide the following information:

• confluent-kafka-python: ('0.11.5', 722176)
• librdkafka: ('0.11.5', 722431)
• Apache Kafka broker version: confluentinc/cp-enterprise-kafka:5.0.0
• Operating system: macOS, 10.13.6, Docker for Mac 18.06.1-ce-mac74

[jschulenklopper]

I encounter the same problem in another example, https://github.com/confluentinc/cp-docker-images/tree/5.0.x/examples/kafka-single-node.

[rnpridgeon]

With docker you need to be careful about where and how addresses are exposed both within the docker network and from outside of it. I highly recommend reviewing the docker documentation on docker network:

https://docs.docker.com/docker-for-mac/networking/

That said I have covered three common scenarios for you below.

1. From the host machine (not in docker):
   p = Producer({'bootstrap.servers': 'localhost:29092'})

2. From a docker container included in compose network:
   p = Producer({'bootstrap.servers': 'broker'})

   or

   p = Producer({'bootstrap.servers': 'broker:9092'})

   The run the container from within the docker network:
   docker run --network

3. From a docker container not included in the compose network:

p = Producer({'bootstrap.servers': 'localhost:29092'})

Then run the container using the host network:
docker run --network host

[rnpridgeon]

Forgot to escape <> for <compose network> <container name> for step 2 and <container name> for step 3. Sorry for any confusion there.