Inconsistent behaviour of docker pull rate limit

[slittle1]

Description

As i understand it, the rate limit on anonymous pulls does not apply to open source project images.
I have verified this using the docker hub api and querying rate limit before and after pulling an alpine linux image, i.e. no change.

However when I conduct the same experiment from within a container, running on the same host and ‘billing’ to the same IP, the rate limit is decremented. If I continue to pull more OS images, I can drive the limit to zero and I get “toomanyrequests: You have reached your pull rate limit…”

Returning to the host, the rate limit also shows as zero, but I can pull the same open source image successfully as expected.

Why does the ‘Open Source’ nature of the pulled image not recognized from within the container?

Docker version 23.0.5, build bc4487a on both host and within container

OS: Debian bullseye

The container is using the docker socket from the host.

Reproduce

limit () {     local TOKEN="";     local LIMIT_DATA="";     local LIMIT="";     local LIMIT_SOURCE=""     TOKEN=$(curl "https://auth.docker.io/token?service=registry.docker.io&scope=repository:ratelimitpreview/test:pull" 2> /dev/null | jq -r .token);     LIMIT_DATA="$(curl --head -H "Authorization: Bearer $TOKEN" https://registry-1.docker.io/v2/ratelimitpreview/test/manifests/latest 2> /dev/null | grep ratelimit)";     LIMIT=$(echo "$LIMIT_DATA" | grep ratelimit-remaining | sed "s#ratelimit-remaining: ##" | cut -d ';' -f 1);     LIMIT_SOURCE=$(echo "$LIMIT_DATA" | grep docker-ratelimit-source | sed "s#docker-ratelimit-source: ##" | cut -d ';' -f 1);     echo "$LIMIT from $LIMIT_SOURCE"; }

Do not login, use anonymous pulls ...

limit
docker pull alpine
limit

IF run under debian bullseye on host, limit does not decrement.
If run under debian bullseye docker container limit does decrement.
Both are billed to the same ip address.

If a large enough number of images from within the container the limit drops to zero.

The next pull of an opensource qualified image will succeed from host, but fail from container.

Expected behavior

the docker pull limit does not decrement when run on a qualified open source image, not matter if the command is run inside of a container or not.

docker version

Client: Docker Engine - Community
 Version:           23.0.5
 API version:       1.42
 Go version:        go1.19.8
 Git commit:        bc4487a
 Built:             Wed Apr 26 16:17:45 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          23.0.5
  API version:      1.42 (minimum version 1.12)
  Go version:       go1.19.8
  Git commit:       94d3ad6
  Built:            Wed Apr 26 16:17:45 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.20
  GitCommit:        2806fc1057397dbaeefbea0e4e17bddfbd388f38
 runc:
  Version:          1.1.5
  GitCommit:        v1.1.5-0-gf19387a
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.4
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.17.3
    Path:     /usr/libexec/docker/cli-plugins/docker-compose
  scan: Docker Scan (Docker Inc.)
    Version:  v0.23.0
    Path:     /usr/libexec/docker/cli-plugins/docker-scan

Server:
 Containers: 48
  Running: 41
  Paused: 0
  Stopped: 7
 Images: 130
 Server Version: 23.0.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2806fc1057397dbaeefbea0e4e17bddfbd388f38
 runc version: v1.1.5-0-gf19387a
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.10.0-21-amd64
 Operating System: Debian GNU/Linux 11 (bullseye)
 OSType: linux
 Architecture: x86_64
 CPUs: 72
 Total Memory: 125.8GiB
 Name: yow2-wrcp2-lx
 ID: fb802abb-6ed1-4880-a801-f587d994ff9f
 Docker Root Dir: /localdisk/var_lib_docker
 Debug Mode: false
 Username: slittlewrs
 Registry: https://index.docker.io/v1/
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional Info

No response

[thaJeztah]

Returning to the host, the rate limit also shows as zero, but I can pull the same open source image successfully as expected.

Is your host a local machine or GitHub actions? GitHub actions sets up authentication with a token as default, which makes it not have rate-limits.

[slittle1]

It was a local machine, and a container running on the same machine. both relied on anonymous pulls. Both are 'charged' to the same IP.