New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Swarm mode with mode=host and port is already in use by service on another host. #31249

Closed
tlvenn opened this Issue Feb 22, 2017 · 18 comments

Comments

Projects
None yet
@tlvenn

tlvenn commented Feb 22, 2017

Hi,

I am using docker swarm mode (1.13) and I have 2 services A and B.
Both services publish ports using exclusively mode=host and both services have placement constraint on a different server.

Now the issue is that both services listen of port 443. Once the first service starts, the second is failing to start with the following error:

Error response from daemon: rpc error: code = 3 desc = port '443' is already in use by service 'A' (pjo2sx4wx3apaejqhgevpkvqs)

I am guessing there is a check somewhere that does not take into account that service A is using mode=host and so it's not mutually exclusive to run services on the same port anymore as far as the services are not running on the same host.

docker version

Client:
 Version:      1.13.0
 API version:  1.25
 Go version:   go1.7.3
 Git commit:   49bf474
 Built:        Tue Jan 17 09:58:26 2017
 OS/Arch:      linux/amd64

Server:
 Version:      1.13.0
 API version:  1.25 (minimum version 1.12)
 Go version:   go1.7.3
 Git commit:   49bf474
 Built:        Tue Jan 17 09:58:26 2017
 OS/Arch:      linux/amd64
 Experimental: false
@aboch

This comment has been minimized.

Contributor

aboch commented Feb 22, 2017

I am guessing there is a check somewhere that does not take into account that service A is using mode=host

Right. checkPortConflicts() must be changed to take that into account.

@ginjo

This comment has been minimized.

ginjo commented Feb 23, 2017

I also tripped over this issue. Same scenario as the OP (also docker 1.13). I'm trying to publish a service port with mode=host, on a single worker node. However once the port has been published on the manager node, I get the port-already-in-use error. I'm looking for workarounds, other than manually starting up the service with docker run, but not finding anything yet.

@bhdrk

This comment has been minimized.

bhdrk commented Feb 24, 2017

I have same issue.

@thaJeztah thaJeztah added the kind/bug label Feb 24, 2017

@thaJeztah thaJeztah added this to the 17.03.0 milestone Feb 24, 2017

@ginjo

This comment has been minimized.

ginjo commented Mar 14, 2017

Any updates on this? I tried version 17.04.0-dev, build 3fe2730, but I'm still getting port-in-use error. Happy to help if there's anything I can do...

@tlvenn

This comment has been minimized.

tlvenn commented Mar 15, 2017

@thaJeztah any chance to get someone from Docker assigned to this issue ? As pointed out by @aboch , the function that need to be patched is checkPortConflicts()

@thaJeztah thaJeztah added this to backlog in maintainers-session Mar 21, 2017

@erinnovations

This comment has been minimized.

erinnovations commented Apr 26, 2017

+1, same issue here

@ilp1995

This comment has been minimized.

ilp1995 commented May 3, 2017

+1, some issue here too!!!
FATA[0010] The following required ports are already in use on your host - 443.

@aaronlehmann

This comment has been minimized.

Contributor

aaronlehmann commented May 11, 2017

Opened docker/swarmkit#2177 for this

@ginjo

This comment has been minimized.

ginjo commented May 13, 2017

Ok newb question here. I see that this issue appears to be fixed in swarmkit - very exciting! Is there a way I can try the fix in docker swarm-mode? I downloaded the latest docker master binaries, however I don't think swarmkit fixes are merged in yet. Or... is swarmkit a separate binary that needs to be updated along with docker? Sorry, I'm a little confused as to how swarmkit fits in with docker.
Thanks!

Edit: sorry... I meant "Moby"

@thaJeztah

This comment has been minimized.

Member

thaJeztah commented May 13, 2017

@ginjo swarmkit is used as a package/component in this repository and source files are vendored in; see #33170 which pulls in the changes that fixed this issue

It las not merge yet

@ginjo

This comment has been minimized.

ginjo commented May 13, 2017

@thaJeztah Thanks, that makes perfect sense.

@aaronlehmann

This comment has been minimized.

Contributor

aaronlehmann commented May 15, 2017

@ginjo: It has been merged now.

@thaJeztah thaJeztah removed this from backlog in maintainers-session May 15, 2017

@meyskens

This comment has been minimized.

meyskens commented May 25, 2017

Has anybody got this to work? I rebuilt from the master and still have the error. I only deployed the dev version to one host. Should I deploy to all?
EDIT: stupid me, i did not deploy it on the leader
EDIT 2: It works 👍

@ginjo

This comment has been minimized.

ginjo commented May 25, 2017

Yes, works great using master binaries from 5/22 👍

@Goram

This comment has been minimized.

Goram commented Jun 20, 2017

Will this come in 17.06 and when?

@aaronlehmann

This comment has been minimized.

Contributor

aaronlehmann commented Jun 20, 2017

Yes, it will be part of the 17.06 release, which is currently near the end of the release candidate phase.

@seffyroff

This comment has been minimized.

seffyroff commented Jan 18, 2018

I see the same issue, running docker engine 17.11 on a worker and 17.12 on master. Swarm has a service (portainer:amd64) deployed on port 9000(ingress), and I wish to deploy a service (portainer:arm64) to a specific node using constraint engine.labels.arch == arm64 and port 9000 host network. Is this expected to work? Should I change the service running on ingress to host networking too?

EDIT: Yes, apparently the ingress network requires exclusive use of any port on the network. Switching service to host networking resolves this.

@ryancastle

This comment has been minimized.

ryancastle commented Oct 3, 2018

Using stop-first in the deployment config is apparently fairly important to prevent dead-locks on the port when using host most. Service updates will fail with host-mode port already in use on 1 node when start-first is used.

services:
  ftp:
    image: hut6/vsftpd
    ports:
      - { published: 21, target: 21, mode: host }
    deploy:
      mode: global
      update_config:
        order: stop-first
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment