docker info reports an index server url as a registry url

[x-yuri]

Description

$ docker info | grep Registry:
 Registry: https://index.docker.io/v1/

$ curl -sS https://index.docker.io/v1/ -o /dev/null -w '%{http_code}'
404

IndexServerAddress
string
Default: "https://index.docker.io/v1/"

Address / URL of the index server that is used for image search, and as a default for user authentication for Docker Hub and Docker Cloud.

https://docs.docker.com/engine/api/v1.41/#tag/System/operation/SystemInfo

I don't see how index.docker.io is used for authentication. Can you possibly provide steps to reproduce and some links to the source code?

From what I can see docker uses registry-1.docker.io to contact the registry. And what's used for authentication is auth.docker.io:

$ curl -sS https://registry-1.docker.io/v2/ -o /dev/null -w '%header{www-authenticate}' |& less
Bearer realm="https://auth.docker.io/token",service="registry.docker.io"

Not to mention that to get the registry-1.docker.io value I had to consult the source code.

It's not clear if there's a difference between a docker registry and an index server.

Here the domain is called legacy.

This whole thing is pretty damn confusing. Particularly the fact that docker info calls an index server a registry and provides a broken URL.

Reproduce

$ docker info | grep Registry:

Expected behavior

docker info shouldn't confuse users. It should be documented what's the difference between an index server and a docker registry. And if there's a difference, docker info shouldn't call an index server a registry. Especially provide a broken URL. Is it even used these days?

docker version

Client:
 Version:           20.10.18
 API version:       1.41
 Go version:        go1.19.1
 Git commit:        b40c2f6b5d
 Built:             Thu Sep 29 08:07:17 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.18
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.19.1
  Git commit:       e42327a6d3
  Built:            Thu Sep 29 08:07:16 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.6.8
  GitCommit:        9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6.m
 runc:
  Version:          1.1.4
  GitCommit:        
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  compose: Docker Compose (Docker Inc., 2.11.1)

Server:
 Containers: 117
  Running: 0
  Paused: 0
  Stopped: 117
 Images: 1165
 Server Version: 20.10.18
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: false
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6.m
 runc version: 
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.19.11-arch1-1
 Operating System: Arch Linux
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 15.02GiB
 Name: yuri2
 ID: GDZF:WOJX:D4NW:H545:6ACG:PLQT:M3CQ:4QKD:TD2V:T5LW:HSBG:QHBC
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional Info

No response

[thaJeztah]

Yes, this is really confusing, and something that will be addressed at some point; there are plans to migrate some domains and update logic around this, but .. this may take some time.

For a bit of background;

index.docker.io was the name for the first "Docker Hub", which was named "the docker index" (hence index.docker.io). This registry used "v1" of the registry specification, which included "search". The Docker Index became "Docker Hub", and the registry v2 API, which was the foundation for the OCI Distribution Spec. The Docker Hub registry migrated to another domain (registry-1.docker.io), however the v2 specification (by design) does not provide search endpoints, so those endpoints still use the v1 API (accessible at https://index.docker.io/v1/search/)

Unfortunately at some point, logic was implemented in the Docker Engine and Docker CLI code to map domains to their new locations (e.g. there's mapping for docker.io/xxxx image references to registry-1.docker.io, as well as authentication mapping from index.docker.io); the same logic also made its way into all container runtimes (containerd, cri-o, kubernetes), which means that, with the Docker Engine only already having 30+ million monthly installs, it's become a bit of a challenge to change these things without breaking existing installations.

[x-yuri]

Thanks for the explanations. So the docker index used registry v1, which had a search route. Docker Hub uses registry v2 at registry-1.docker.io, but the obsolete routes are still available at index.docker.io?.. And an index server is an obsolete concept as well?

[thaJeztah]

The Docker Hub registry (currently) runs at registry-1.docker.io, which is compliant with the OCI Distribution Spec.

Not sure if "obsolete" is the correct term to use for the index.docker.io search endpoints, but the OCI Specification does not include search functionality, leaving it up to implementations wether or not they provide a search feature (some of the public registries decided not to).

Given that there's no standardised replacement for search, some registries still provide the v1 search, conforming to the Docker Registry V1 "search" specification (this includes Docker Hub and many self-hosted registries that deploy the open-source CNCF distribution registry (https://github.com/distribution/distribution).

[x-yuri]

Okay, so Docker Index is the previous name of Docker Hub. They are names of the website that includes a docker registry (registry-1.docker.io) and an index server (index.docker.io). An index server is a service that allows to search the registry.

this includes Docker Hub and many self-hosted registries that deploy the open-source CNCF distribution registry (https://github.com/distribution/distribution)

At first glance it doesn't look like distribution/distribution provides a search endpoint. Am I wrong? That means they're running a separate service (an index server) that provides the search endpoint?

[thaJeztah]

At first glance it doesn't look like distribution/distribution provides a search endpoint. Am I wrong? That means they're running a separate service (an index server) that provides the search endpoint?

Oh, good call, I may be mistaken and distribution/distribution may indeed not be providing that endpoint. I know Docker Hub runs a separate service for it (but wasn't 100% sure if the open-source registry still included it)

I opened a PR to remove this field from the default docker info output;

• docker info: don't print "registry" (IndexServerAddress) #4204