docker-compose run fails for running network_mode=host services

[SpComb]

For a network_mode: host service,

version: '2'
services:
  redis:
    image: redis
    network_mode: host

$ docker-compose version
docker-compose version 1.11.1, build 7c5d5e4
docker-py version: 2.0.2
CPython version: 2.7.13
OpenSSL version: OpenSSL 1.0.1t  3 May 2016
$ hostname
localhost
$ docker-compose run --rm redis hostname
localhost

The docker-compose run command fails if the service is running:

$ docker-compose up -d redis
Starting composetest_redis_1
$ docker-compose run --rm redis hostname
ERROR: Cannot create container for service redis: Conflicting options: host type networking can't be used with links. This would result in undefined behavior
$ docker-compose stop redis         
Stopping composetest_redis_1 ... done
$ docker-compose run --rm redis hostname
localhost

It looks like docker-compose --verbose run tries to do some --link trickery against the running container, but this is not compatible with the --net host:

compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [],
 'Links': ['composetest_redis_1:composetest_redis_1',
           'composetest_redis_1:redis',
           'composetest_redis_1:redis_1'],
 'LogConfig': {'Config': {}, 'Type': u''},
 'NetworkMode': 'host',
 'PortBindings': {},
 'VolumesFrom': []}

ERROR: compose.cli.main.main: Cannot create container for service redis: Conflicting options: host type networking can't be used with links. This would result in undefined behavior

Possibly related: #2480

[sivabudh]

I'm also running into the exact same problem. Here are my versions:

docker-compose version 1.11.2, build dfed245
Docker version 17.03.1-ce, build c6d412e

and here's the gist of my docker-compose.yml file:

(some part redacted)

version: '3'

services:

  backend:
    container_name: backend
    build:
      context: .
      dockerfile: DockerfileBackend
    image: 123456.ecr.some-region.amazonaws.com/some-repo:latest
    environment:
      - DATABASE_URL=postgres://postgres:P@ssw0rd@localhost:5432/some_db
    volumes:
      - ./frontend_assets/static_files:/code/static_files
    command: gunicorn --bind 0.0.0.0:8000 --workers 3 --worker-class gevent app.config.wsgi:application --log-level=INFO
    ports:
      - "8000:8000"
    network_mode: "host"

  frontend:
    container_name: frontend
    build:
      context: .
      dockerfile: DockerfileFrontend
    image: 123456.dkr.ecr.some-region.amazonaws.com/frontend-repo:latest
    volumes:
      - ./frontend_assets:/code/frontend_assets

With this docker-compose.yml, the error will happen when the backend container is already running and we attempt to execute another docker run command related to backend.

Here's a concrete example. Assuming that backend is already running, this is the behavior:

# When running `frontend` container, there's no problem
docker-compose run frontend cp -rf /code/static_files/ /code/frontend_assets/
# But once you run the `backend` container while there's already one running...
docker-compose run backend python manage.py collectstatic --noinput

You will immediately see this error:

ERROR: Cannot create container for service backend: Conflicting options: host type networking can't be used with links. This would result in undefined behavior

---

Workaround

There's a workaround for this issue that works for me, which is to do a docker-compose down prior to doing docker-compose run backend. Eg, this is what I do now:

docker-compose down
docker-compose run frontend cp -rf /code/static_files/ /code/frontend_assets/
# No problem. We are happy! ^_^
docker-compose run backend python manage.py collectstatic --noinput

[usmanm]

Running into the same issue. There should be a way to bypass the automatic linking that compose is trying to do. The workaround that @sivabudh describes isn't really ideal if the command being run in the second step takes a non-trivial amount of time.

[sivabudh]

Agreed with @usmanm on his comments. Fortunately, my app update takes roughly 6 seconds, and we are lucky that our apps can have daily maintenance window.

[ghost]

This is still relevant 👍

[damouse]

Same issue.

[tomholub]

Encountered here too

[rhoerbe]

Same issue

[rhoerbe]

likewise

[rhoerbe]

The reason turned out to be a misleading error message in my case. An instance of the container was in restart mode, and docker compose got confused (I have not links, but network_mode=host)

[thomas-profitt]

This is still reproducible in docker-compose version 1.22.0, build f46880f.

docker-compose run some-container any-command, where some-container is any container that's got network_mode: host and is currently Restarting or Up.

[nathantsoi]

same here. this is a problem when i bring up an array of services with docker-compose up, one or more of which is designed to be interactive and i need to run docker-compose run [interactive service] bash -l to enter that particular container

[DavidGamba]

The same problem occurs during docker-compose build, docker build --network host ... works properly.

[artm]

@nathantsoi docker-compose run ... does not enter the container, it makes a new container with the same settings. To "enter" a running container you would use docker-compose exec.

[hholst80]

Two and a half years later, we still have the same problem.

$ docker-compose version
docker-compose version 1.24.1, build 4667896b
docker-py version: 3.7.3
CPython version: 3.6.8
OpenSSL version: OpenSSL 1.1.0j  20 Nov 2018
$ cat docker-compose.yml
version: '2'

services:
  foo:
    image: debian
    command: sleep inf
    network_mode: host
  bar:
    image: debian
    command: sleep inf
    network_mode: host
$ docker-compose up -d
docker-compose_bar_1 is up-to-date
docker-compose_foo_1 is up-to-date
$ docker-compose run --rm bar echo hello world
ERROR: Cannot create container for service bar: b"conflicting options: host type networking can't be used with links. This would result in undefined behavior"
$ 

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[rhoerbe]

The problem does not go away by doing nothing, then flagging it as stale, and finally closing the ticket. Bugs need to be fixed, not managed

[stale]

This issue has been automatically marked as not stale anymore due to the recent activity.

[PawelAdamczuk]

I encountered the same problem.

[traviswaelbro]

I am also encountering the same problem on a container with network_mode: host.

As a workaround, I had to stop my running containers, then run my separate run command.

$ docker-compose up -d
docker-compose_bar_1 is up-to-date
docker-compose_foo_1 is up-to-date
$ docker-compose run --rm bar echo hello world
ERROR: Cannot create container for service bar: conflicting options: host type networking can't be used with links. This would result in undefined behavior
$ docker-compose stop
$ docker-compose run --rm bar echo hello world
hello world
$

[shomeax]

I got the same issue. Trying to run a one-off command to service previously started via docker-compose with network_mode: host fails with error.

# docker-compose --version
docker-compose version 1.25.0, build 0a186604
#  docker-compose run webserver ps
ERROR: Cannot create container for service webserver: conflicting options: host type networking can't be used with links. This would result in undefined behavior

verbose output:

compose.config.config.find: Using configuration files: /home/project/frontend-docker-compose-start-config.latest
docker.utils.config.find_config_file: Trying paths: ['/home/.docker/config.json', '/home/.dockercfg']
docker.utils.config.find_config_file: Found file at path: /home/.docker/config.json
docker.auth.load_config: Found 'auths' section
docker.auth.parse_auth: Found entry (registry='****', username='AWS')
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.38/version HTTP/1.1" 200 567
compose.cli.command.get_client: docker-compose version 1.25.0, build 0a186604
docker-py version: 4.1.0
CPython version: 3.7.4
OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019
compose.cli.command.get_client: Docker base_url: http+docker://localhost
compose.cli.command.get_client: Docker version: Platform={'Name': ''}, Components=[{'Name': 'Engine', 'Version': '18.09.9-ce', 'Details': {'ApiVersion': '1.39', 'Arch': 'amd64', 'BuildTime': '2019-11-01T19:28:24.000000000+00:00', 'Experimental': 'false', 'GitCommit': '039a7df', 'GoVersion': 'go1.10.3', 'KernelVersion': '4.14.146-120.181.amzn2.x86_64', 'MinAPIVersion': '1.12', 'Os': 'linux'}}], Version=18.09.9-ce, ApiVersion=1.39, MinAPIVersion=1.12, GitCommit=039a7df, GoVersion=go1.10.3, Os=linux, Arch=amd64, KernelVersion=4.14.146-120.181.amzn2.x86_64, BuildTime=2019-11-01T19:28:24.000000000+00:00
compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('project_default')
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.38/networks/project_default HTTP/1.1" 404 55
compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('project_default')
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.38/networks/project_default HTTP/1.1" 200 822
compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': True,
 'ConfigFrom': {'Network': ''},
 'ConfigOnly': False,
 'Containers': {'5079456ae0bf67c3eddfec0dc888bb6926a1d7c97a18515260582f765297cc17': {'EndpointID': 'b89c45692b6a4ab4106111185474b03363e0bae22ddbe95689e0d91da225fac5',
                                                                                     'IPv4Address': '192.168.240.2/20',
                                                                                     'IPv6Address': '',
                                                                                     'MacAddress': '02:42:c0:a8:f0:02',
                                                                                     'Name': 'config'}},
 'Created': '2020-04-07T15:56:31.879957006Z',
 'Driver': 'bridge',
...
compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('****:master')
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.38/images/****:master/json HTTP/1.1" 200 None
compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64',
 'Author': '',
 'Comment': 'buildkit.dockerfile.v0',
 'Config': {'ArgsEscaped': True,
            'AttachStderr': False,
            'AttachStdin': False,
            'AttachStdout': False,
            'Cmd': ['/bin/sh',
                    '-c',
                    '/bin/sh -c "envsubst \'${SERVER_NAME}\' < '
...
compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=project', 'com.docker.compose.service=webserver', 'com.docker.compose.oneoff=False']})
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.38/containers/json?limit=-1&all=0&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dproject%22%2C+%22com.docker.compose.service%3Dwebserver%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D HTTP/1.1" 200 1596
compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 1 items)
compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('db2fcd900b77b77342a44535fbc063d93609bb43467b21e3c37678fe9db235f5')
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.38/containers/db2fcd900b77b77342a44535fbc063d93609bb43467b21e3c37678fe9db235f5/json HTTP/1.1" 200 None
compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '',
 'Args': ['-c',
          '/bin/sh -c "envsubst \'${SERVER_NAME}\' < '
          '/etc/nginx/conf.d/*** > /etc/nginx/conf.d/server.conf '
          '&& exec nginx -g \'daemon off;\'"'],
 'Config': {'ArgsEscaped': True,
            'AttachStderr': False,
            'AttachStdin': False,
            'AttachStdout': False,
            'Cmd': ['/bin/sh',
...
compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[('webserver', 'webserver')], port_bindings={}, binds=['/etc/ssl/certs/****:/etc/ssl/certs/****:rw'], volumes_from=[], privileged=False, network_mode='host', devices=None, dns=None, dns_opt=None, dns_search=None, restart_policy=None, runtime=None, cap_add=None, cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': 'json-file', 'Config': {'max-file': '10', 'max-size': '200k'}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=None, ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=None, device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None)
compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': ['/etc/ssl/certs/****:/etc/ssl/certs/****:rw'],
 'Links': ['webserver:webserver'],
 'LogConfig': {'Config': {'max-file': '10', 'max-size': '200k'},
               'Type': 'json-file'},
 'NetworkMode': 'host',
 'PortBindings': {},
 'VolumesFrom': []}
compose.cli.verbose_proxy.proxy_callable: docker create_container <- (environment=[], image='****', volumes={'/etc/ssl/certs/***': {}}, command=['ps'], tty=True, stdin_open=True, detach=False, ports=[], name='project_webserver_run_1ce47b915946', labels={'com.docker.compose.project': 'project', 'com.docker.compose.service': 'webserver', 'com.docker.compose.oneoff': 'True', 'com.docker.compose.project.working_dir': '/home/project', 'com.docker.compose.project.config_files': '/home/project/frontend-docker-compose-start-config.latest', 'com.docker.compose.slug': '1ce47b915946136712ea2c7d0731230e1223b6b90cd273a6509286fb1f58936', 'com.docker.compose.version': '1.25.0'}, host_config={'NetworkMode': 'host', 'VolumesFrom': [], 'Binds': ['/etc/ssl/certs/***:/etc/ssl/certs/***:rw'], 'PortBindings': {}, 'Links': ['webserver:webserver'], 'LogConfig': {'Type': 'json-file', 'Config': {'max-file': '10', 'max-size': '200k'}}})
urllib3.connectionpool._make_request: http://localhost:None "POST /v1.38/containers/create?name=project_webserver_run_1ce47b915946 HTTP/1.1" 400 122

Otherwise speaking, compose is trying to create a temporary container for a one-off command and link it to the existing container. Proper fix could be passing --network-mode host to a run command and don't use the link when it is passed.

Workaround: don't mess with the run, use exec:

PID   USER     TIME  COMMAND
    1 root      0:00 nginx: master process nginx -g daemon off;
    7 nginx     0:33 nginx: worker process
    8 nginx     0:00 nginx: worker process
   14 root      0:00 sh
   39 root      0:00 ps

[penkong]

with
docker build -t <image name> --network=host
it work fine but with docker-compose up --build does not work.
all top hints already tried.

[matthew-nm]

Adding my comment to prevent bot from marking as stale. I encountered this issue as well.

Docker version 19.03.8, build afacb8b
Had container "up" after docker-compose up -d.
Attempted to execute docker-compose run <service> <command> and obtained same error message as OP.

Had to bring down container that was "up" before executing run command.

[Skydev0h]

I workarounded it (at least for openvpn) by creating a sibling container with -cmd prefix without network mode host.
Can be useful to execute commands that modify attached volume and do not require other quirks.

Example:

version: '2'
services:
  openvpn:
    cap_add:
     - NET_ADMIN
    image: kylemanna/openvpn
    container_name: openvpn
    ports:
     - "1194:1194/udp"
    restart: always
    volumes:
     - ./openvpn-data/conf:/etc/openvpn
    network_mode: host
    pid: host
  openvpn-cmd:
    image: kylemanna/openvpn
    container_name: openvpn-cmd
    volumes:
     - ./openvpn-data/conf:/etc/openvpn

And instead of docker-compose run --rm openvpn easyrsa build-client-full my.client.com I do docker-compose run --rm openvpn-cmd easyrsa build-client-full my.client.com.

[ivictbor]

Workaround works but I am using run to prevent downtime. After switching to host mode due to performance considerations run command is useless.

[MichaelMackus]

You should be able to also do this if your command is simple enough:

docker run -it $(docker-compose images -q SERVICE) COMMAND

Where SERVICE is the service in question in your docker-compose.yml file and COMMAND is the command you want to run. You'll have to add all the relevant args to docker run manually, though.

It would be nice to be able to run services with host mode networking, even if the networking part were disabled.

[JanJamaszyk-FlyNow]

I am also seeing the same issue in a different use-case, is there any way to disable the links docker-compose creates automatically?
What is the way forward on this issue? It is open since 3 years, is there any way we can escalate this?

[bierzorutas]

Same problem here ... unbelievable how after years this is still present... using docker-compose up

[rkmfl]

Have you tried changing the version in the docker-config.yml to 3.4?

version: "3.4"
#...trimmed...
network_mode: host

This appears to have fixed several issues for me, including the host-mode.

[chrisawad]

Using syntax version 3.7 and just came across this also.

Like everyone has already explained, this only happens when there's an existing container running, so my suggestion is to use docker exec to execute in the already running container instead of running a new one.

[MichaelAkvo]

This is also happening in https://github.com/akvo/akvo-rsr where network_mode: service:main_network is being used.

$ docker-compose up -d
Creating akvo-rsr_rsr-memcached_1 ... done
Creating akvo-rsr_rsrdbhost_1     ... done
Creating akvo-rsr_mainnetwork_1   ... done
Creating akvo-rsr_web_1           ... done
Creating akvo-rsr_reports_1       ... done
Creating akvo-rsr_nginx_1         ... done
$ docker-compose run --rm web echo lol
Starting akvo-rsr_mainnetwork_1 ... done
ERROR: Cannot create container for service web: conflicting options: container type network can't be used with links. This would result in undefined behavior

[Dens49]

Still relevant. I wanted to use docker-compose run in order to prevent downtime. So having two containers of that service running at the same time was my goal.
The workaround of @MichaelMackus works for me but you have to make sure to set up the docker run command properly which is way more error prone.
Would be nice to be able to keep it all in docker-compose.

[ndeloof]

Docker compose v2 doesn't suffer this limitation

$docker compose version
Docker Compose version v2.4.1
$ docker compose run --rm redis hostname
docker-desktop

[glours]

I close the issue, Compose v2 works as expected and Compose v1 is end-of-life
Tested with @hholst80 sample

> bat compose.yaml
───────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: compose.yaml
       │ Size: 152 B
───────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ services:
   2   │   foo:
   3   │     image: debian
   4   │     command: sleep inf
   5   │     network_mode: host
   6   │   bar:
   7   │     image: debian
   8   │     command: sleep inf
   9   │     network_mode: host
───────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
> docker compose up -d
[+] Running 3/3
 ⠿ foo Pulled                                                                                                                                                                                                                            5.6s
   ⠿ cfc947b533a3 Pull complete                                                                                                                                                                                                          3.2s
 ⠿ bar Pulled                                                                                                                                                                                                                            5.6s
[+] Running 2/2
 ⠿ Container issue-4548-foo-1  Started                                                                                                                                                                                                   0.4s
 ⠿ Container issue-4548-bar-1  Started

[SeLub]

Docker Compose version v2.10.2

Error response from daemon: conflicting options: host type networking can't be used with links. This would result in undefined behavior

  api:
    image: grabli
    restart: on-failure
    container_name: api
    build:
      context: .
    env_file: docker-compose.env
    environment:
      SERVICES: api
      PORT: 3001
    ports:
      - 3001:3001
    links:
      - mqtt
    depends_on:
      - mqtt
      - googleOauth2
    network_mode: host

---

network_mode: "host" cannot be mixed with links.

Solution:
remove

    links:
      - mqtt
     

[simwai]

Docker compose v2 doesn't suffer this limitation

$docker compose version
Docker Compose version v2.4.1
$ docker compose run --rm redis hostname
docker-desktop

Unfortunately, it does. This problem still occurs on v2 and v3 and v3.4.

[ndeloof]

@simwai version in compose file is unrelated (and obsolete)
I can't reproduce issue using latest docker compose release, please provide a reproduction example if you do

[hholst80]

Works just fine with latest Compose spec and Docker Engine 26.1.0:

# cat compose.yaml 
services:
  foo:
    image: debian
    command: sleep inf
    network_mode: host
  bar:
    image: debian
    command: sleep inf
    network_mode: host