Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot push image to local registry #3611

Closed
NFarrington opened this issue Apr 3, 2019 · 6 comments

Comments

@NFarrington
Copy link

commented Apr 3, 2019

  • I have tried with the latest version of my channel (Stable or Edge)
  • I have uploaded Diagnostics
  • Diagnostics ID: 5B25BFA5-45C6-4E9C-9CCF-31B8375E0A19/20190403095701

Expected behavior

When following the instructions to run a local registry at https://docs.docker.com/registry/deploying/#run-a-local-registry, you should be able to push an image to a local registry.

Actual behavior

When following the documentation, pushing to a local registry now fails with the error Get http://localhost:5000/v2/: dial tcp 127.0.0.1:5000: connect: connection refused:

$ docker run -d -p 5000:5000 --restart=always --name registry registry:2
58c57b9042a75706167ef5eb458fb094dec247fed8b509085c186a6ebce23af9
$ docker pull ubuntu:16.04
16.04: Pulling from library/ubuntu
Digest: sha256:58d0da8bc2f434983c6ca4713b08be00ff5586eb5cdff47bcde4b2e88fd40f88
Status: Image is up to date for ubuntu:16.04
$ docker tag ubuntu:16.04 localhost:5000/my-ubuntu
$ docker push localhost:5000/my-ubuntu
The push refers to repository [localhost:5000/my-ubuntu]
Get http://localhost:5000/v2/: dial tcp 127.0.0.1:5000: connect: connection refused

Information

  • macOS Version: 10.14.4

After upgrading from Docker for Mac 2.0.2.1 to 2.0.3.0, Docker is now unable to push images to a local registry via localhost, which should be possible based on https://docs.docker.com/registry/deploying/#run-a-local-registry. I would assume this has been caused by the fixes that were applied in 2.0.3.0 for #3522

Diagnostic logs

Docker for Mac: version 2.0.3.0

Steps to reproduce the behavior

To reproduce, run the following using version 2.0.3.0:

docker run -d -p 5000:5000 --restart=always --name registry registry:2
docker pull ubuntu:16.04
docker tag ubuntu:16.04 localhost:5000/my-ubuntu
docker push localhost:5000/my-ubuntu
@jldec

This comment has been minimized.

Copy link

commented Apr 3, 2019

We're seeing the same problem (just on Edge v2.0.3.0, not on stable v2.0.0.3) on both Mac and Windows.

The issue manifests if you use an image tag localhost:5000/... or 127.0.0.1:5000/... but not if you use a different hostname e.g. registry.me pointed to localhost via /etc/hosts

$ cat /etc/hosts
##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting.  Do not change this entry.
##
127.0.0.1	      localhost registry.me
255.255.255.255	broadcasthost
::1             localhost
$ docker run -d -p 5000:5000 --restart=always --name registry registry:2
Unable to find image 'registry:2' locally
2: Pulling from library/registry
c87736221ed0: Pull complete 
1cc8e0bb44df: Pull complete 
54d33bcb37f5: Pull complete 
e8afc091c171: Pull complete 
b4541f6d3db6: Pull complete 
Digest: sha256:3b00e5438ebd8835bcfa7bf5246445a6b57b9a50473e89c02ecc8e575be3ebb5
Status: Downloaded newer image for registry:2
67969b56d8d33056e8d66b1e7cb2e3e01fd79fa3b1af7fa5ed8597ea96cb3924

$ docker pull ubuntu:16.04
16.04: Pulling from library/ubuntu
34667c7e4631: Pull complete 
d18d76a881a4: Pull complete 
119c7358fbfc: Pull complete 
2aaf13f3eff0: Pull complete 
Digest: sha256:58d0da8bc2f434983c6ca4713b08be00ff5586eb5cdff47bcde4b2e88fd40f88
Status: Downloaded newer image for ubuntu:16.04

$ docker tag ubuntu:16.04 localhost:5000/my-ubuntu

$ docker push localhost:5000/my-ubuntu
The push refers to repository [localhost:5000/my-ubuntu]
Get http://localhost:5000/v2/: dial tcp [::1]:5000: connect: connection refused

$ docker tag ubuntu:16.04 127.0.0.1:5000/my-ubuntu

$ docker push 127.0.0.1:5000/my-ubuntu
The push refers to repository [127.0.0.1:5000/my-ubuntu]
Get http://127.0.0.1:5000/v2/: dial tcp 127.0.0.1:5000: connect: connection refused

$ docker tag ubuntu:16.04 registry.me:5000/my-ubuntu

$ docker push registry.me:5000/my-ubuntu
The push refers to repository [registry.me:5000/my-ubuntu]
297fd071ca2f: Pushed 
2f0d1e8214b2: Pushed 
7dd604ffa87f: Pushed 
aa54c2bc1229: Pushed 
latest: digest: sha256:d7d4f38deab29555ed2a9f13f4cb71f33e6f8788529512401d4e0638f2c3ba51 size: 1150

note i have those registry names configured as insecure

$ cat ~/.docker/daemon.json 
{
  "insecure-registries" : [
    "localhost:5000",
    "127.0.0.1:5000",
    "registry.me:5000"
  ],
  "debug" : true,
  "experimental" : true
}

cc: @glyn

@jldec

This comment has been minimized.

Copy link

commented Apr 3, 2019

b.t.w the fact that the error messages are slightly different for the different hostnames suggests that maybe this is a problem in the way the daemon is looking up hostnames via DNS.

Get http://localhost:5000/v2/: dial tcp [::1]:5000: connect: connection refused
Get http://127.0.0.1:5000/v2/: dial tcp 127.0.0.1:5000: connect: connection refused
@moberg

This comment has been minimized.

Copy link

commented Apr 24, 2019

I have the exact same issue.

djs55 added a commit to djs55/vpnkit that referenced this issue May 1, 2019

vpnkit-userland-proxy: bind ports in the VM on a best-effort basis
When Docker attempts to expose a port on the host with

  docker run -p <external>:<internal>

We have to decide whether to also bind the same port in the VM. In
the case of addresses like `0.0.0.0:80` or `127.0.0.1:80` this can
make sense, and it will allow running a Docker registry in a container
and pushing directly to it, see [docker/for-mac#3611]

However it also opens us up to accidental port clashes between the
user's ports and any that we have allocated internally. Recently this
happened when a compose on kubernetes container was run with `--net=host`
and bound port 8080.

This patch adds support for "best-effort" binding in the VM and makes this
the default. This should re-enable the registry use-case while making us
robust to the compose on kubernetes problem. The only downside is that
if there is a port clash in the VM, the user won't be notified.

Signed-off-by: David Scott <dave.scott@docker.com>

djs55 added a commit to djs55/vpnkit that referenced this issue May 1, 2019

vpnkit-userland-proxy: bind ports in the VM on a best-effort basis
When Docker attempts to expose a port on the host with

  docker run -p <external>:<internal>

We have to decide whether to also bind the same port in the VM. In
the case of addresses like `0.0.0.0:80` or `127.0.0.1:80` this can
make sense, and it will allow running a Docker registry in a container
and pushing directly to it, see [docker/for-mac#3611]

However it also opens us up to accidental port clashes between the
user's ports and any that we have allocated internally. Recently this
happened when a compose on kubernetes container was run with `--net=host`
and bound port 8080, see [docker/compose-on-kubenetes#70].

This patch adds support for "best-effort" binding in the VM and makes this
the default. This should re-enable the registry use-case while making us
robust to the compose on kubernetes problem. The only downside is that
if there is a port clash in the VM, the user won't be notified.

Signed-off-by: David Scott <dave.scott@docker.com>

djs55 added a commit to djs55/vpnkit that referenced this issue May 1, 2019

vpnkit-userland-proxy: bind ports in the VM on a best-effort basis
When Docker attempts to expose a port on the host with

  docker run -p <external>:<internal>

We have to decide whether to also bind the same port in the VM. In
the case of addresses like `0.0.0.0:80` or `127.0.0.1:80` this can
make sense, and it will allow running a Docker registry in a container
and pushing directly to it, see [docker/for-mac#3611]

However it also opens us up to accidental port clashes between the
user's ports and any that we have allocated internally. Recently this
happened when a compose on kubernetes container was run with `--net=host`
and bound port 8080, see [docker/compose-on-kubenetes#70].

This patch adds support for "best-effort" binding in the VM and makes this
the default. This should re-enable the registry use-case while making us
robust to the compose on kubernetes problem. The only downside is that
if there is a port clash in the VM, the user won't be notified.

Signed-off-by: David Scott <dave.scott@docker.com>

djs55 added a commit to djs55/vpnkit that referenced this issue May 1, 2019

vpnkit-userland-proxy: bind ports in the VM on a best-effort basis
When Docker attempts to expose a port on the host with

  docker run -p <external>:<internal>

We have to decide whether to also bind the same port in the VM. In
the case of addresses like `0.0.0.0:80` or `127.0.0.1:80` this can
make sense, and it will allow running a Docker registry in a container
and pushing directly to it, see [docker/for-mac#3611]

However it also opens us up to accidental port clashes between the
user's ports and any that we have allocated internally. Recently this
happened when a compose on kubernetes container was run with `--net=host`
and bound port 8080, see [docker/compose-on-kubenetes#70].

This patch adds support for "best-effort" binding in the VM and makes this
the default. This should re-enable the registry use-case while making us
robust to the compose on kubernetes problem. The only downside is that
if there is a port clash in the VM, the user won't be notified.

Signed-off-by: David Scott <dave.scott@docker.com>

djs55 added a commit to djs55/vpnkit that referenced this issue May 1, 2019

vpnkit-userland-proxy: bind ports in the VM on a best-effort basis
When Docker attempts to expose a port on the host with

  docker run -p <external>:<internal>

We have to decide whether to also bind the same port in the VM. In
the case of addresses like `0.0.0.0:80` or `127.0.0.1:80` this can
make sense, and it will allow running a Docker registry in a container
and pushing directly to it, see [docker/for-mac#3611]

However it also opens us up to accidental port clashes between the
user's ports and any that we have allocated internally. Recently this
happened when a compose on kubernetes container was run with `--net=host`
and bound port 8080, see [docker/compose-on-kubenetes#70].

This patch adds support for "best-effort" binding in the VM and makes this
the default. This should re-enable the registry use-case while making us
robust to the compose on kubernetes problem. The only downside is that
if there is a port clash in the VM, the user won't be notified.

This patch maintains limited backward-compatibility with scripts which have
`-no-local-ip` flags (with no arguments) but removes the argument from
the help text.

Signed-off-by: David Scott <dave.scott@docker.com>

djs55 added a commit to djs55/vpnkit that referenced this issue May 1, 2019

vpnkit-userland-proxy: bind ports in the VM on a best-effort basis
When Docker attempts to expose a port on the host with

  docker run -p <external>:<internal>

We have to decide whether to also bind the same port in the VM. In
the case of addresses like `0.0.0.0:80` or `127.0.0.1:80` this can
make sense, and it will allow running a Docker registry in a container
and pushing directly to it, see [docker/for-mac#3611]

However it also opens us up to accidental port clashes between the
user's ports and any that we have allocated internally. Recently this
happened when a compose on kubernetes container was run with `--net=host`
and bound port 8080, see [docker/compose-on-kubenetes#70].

This patch adds support for "best-effort" binding in the VM and makes this
the default. This should re-enable the registry use-case while making us
robust to the compose on kubernetes problem. The only downside is that
if there is a port clash in the VM, the user won't be notified.

This patch maintains limited backward-compatibility with scripts which have
`-no-local-ip` flags (with no arguments) but removes the argument from
the help text.

Signed-off-by: David Scott <dave.scott@docker.com>
@djs55

This comment has been minimized.

Copy link
Contributor

commented May 3, 2019

Thanks for the report (and for the ping via the Docker booth at DockerCon). The fix is merged in the vpnkit-userland-proxy helper tool which is inside the VM image. I've made a PR to Docker Desktop itself adding the fix, an end-to-end test to prevent this from happening in future and a release note tagging this issue.

@jldec

This comment has been minimized.

Copy link

commented Jun 12, 2019

Confirmed fixed in edge release v2.0.5.0 from June 11. 2019

@djs55

This comment has been minimized.

Copy link
Contributor

commented Jun 12, 2019

@jldec thanks for the confirmation. I'll chose this ticket but feel free to comment again (or make a fresh issue) if there are further problems.

@djs55 djs55 closed this Jun 12, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.