-
Notifications
You must be signed in to change notification settings - Fork 6.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Docker Rate limit on docker.io/library/nginx:1.23.0-alpine imagepull #9548
Comments
In the past, I was able to use docker login. But it seems that is not an option using crictl. I am not able to find documentation/help on how to get around this. Any help would be greatly appreciated. |
This isn't really a bug for kubespray is it? Also, this should help you: |
How to explain this. Ok, on the first run for kubespray, crictl is not yet available, so that link you provided, does not help me, as I have tried that. If I run the playbook the 1st time and it errors out where I am seeing it, then I can use crictl with the creds to pull in that image from the command line, run kubespray again, and it works as expected, since the dockerhub image is already available locally. What I don't see, and I can be missing it, is I was hoping there was a way to set those creds as part of the playbook run, or, for example in the inventory/x/group_vars/all/containerd.yml
Where I can feed in the auth for dockerhub, or through the cli when running the playbook. |
Is |
containerd is the chosen runtime. Initially, without the auth being set in the inventory, I got the docker pull error. I am just not sure what I am missing. Here is my current containerd.yml
|
I'm hitting this same issue. Also tried with config below:
|
I get the same error, If you do |
I think this is a problem kubespray. I have a pro subscription on docker.io, so I have 5000 connections.
|
The Kubernetes project currently lacks enough contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale |
I had the same issue on my side (with Kubespray v2.21.0) and I fixed it with the following:
So it seems that by default Kubespray sets endpoint And the Somebody can confirm? /remove-lifecycle stale |
Same issue here
|
I am hitting this issue as well and specifying containerd registry auth like this:
doesn't seem to help. Who knows how can I redefined the variable to point to a ghcr mirror where nginx is located to alleviate pull rate limits? |
The Kubernetes project currently lacks enough contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale |
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten |
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /close not-planned |
@k8s-triage-robot: Closing this issue, marking it as "Not Planned". In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
printf "$(uname -srm)\n$(cat /etc/os-release)\n"
):Linux 3.10.0-1160.80.1.el7.x86_64 x86_64
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"
CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"
Version of Ansible (
ansible --version
):ansible [core 2.11.11]
config file = /opt/kubespray/ansible.cfg
configured module search path = ['/opt/kubespray/library']
ansible python module location = /usr/local/lib/python3.6/site-packages/ansible
ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/local/bin/ansible
python version = 3.6.8 (default, Nov 16 2020, 16:55:22) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]
jinja version = 2.11.3
libyaml = True
Version of Python (
python --version
):Python 3.6.8
Kubespray version (commit) (
git rev-parse --short HEAD
):Tag: v2.20.0
Output of ansible run:
TASK [download : Set image save/load command for containerd on localhost] ******
ok: [node1]
Tuesday 06 December 2022 22:38:23 +0000 (0:00:00.410) 0:12:37.239 ******
Tuesday 06 December 2022 22:38:23 +0000 (0:00:00.232) 0:12:37.472 ******
TASK [download : download_container | Prepare container download] **************
included: /opt/kubespray/roles/download/tasks/check_pull_required.yml for node1
Tuesday 06 December 2022 22:38:23 +0000 (0:00:00.428) 0:12:37.901 ******
TASK [download : check_pull_required | Generate a list of information about the images on a node] ***
ok: [node1]
Tuesday 06 December 2022 22:38:25 +0000 (0:00:01.895) 0:12:39.796 ******
TASK [download : check_pull_required | Set pull_required if the desired image is not yet loaded] ***
ok: [node1]
Tuesday 06 December 2022 22:38:25 +0000 (0:00:00.337) 0:12:40.134 ******
Tuesday 06 December 2022 22:38:26 +0000 (0:00:00.126) 0:12:40.261 ******
TASK [download : debug] ********************************************************
ok: [node1] => {
"msg": "Pull docker.io/library/nginx:1.23.0-alpine required is: True"
}
Tuesday 06 December 2022 22:38:26 +0000 (0:00:00.183) 0:12:40.444 ******
Tuesday 06 December 2022 22:38:26 +0000 (0:00:00.091) 0:12:40.536 ******
Tuesday 06 December 2022 22:38:26 +0000 (0:00:00.144) 0:12:40.680 ******
Tuesday 06 December 2022 22:38:26 +0000 (0:00:00.169) 0:12:40.849 ******
FAILED - RETRYING: download_container | Download image if required (4 retries left).
FAILED - RETRYING: download_container | Download image if required (3 retries left).
FAILED - RETRYING: download_container | Download image if required (2 retries left).
FAILED - RETRYING: download_container | Download image if required (1 retries left).
TASK [download : download_container | Download image if required] **************
fatal: [node1 -> node1]: FAILED! => {"attempts": 4, "changed": true, "cmd": ["/usr/local/bin/nerdctl", "-n", "k8s.io", "pull", "--quiet", "docker.io/library/nginx:1.23.0-alpine"], "delta": "0:00:00.990523", "end": "2022-12-06 22:38:47.889778", "msg": "non-zero return code", "rc": 1, "start": "2022-12-06 22:38:46.899255", "stderr": "time="2022-12-06T22:38:47Z" level=fatal msg="failed to copy: httpReadSeeker: failed open: unexpected status code https://registry-1.docker.io/v2/library/nginx/manifests/sha256:4a846cc240449c53c8ae24269ba6bcaee5167d8ad75cd2a8d8ba422b7c726979: 429 Too Many Requests - Server message: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit\"", "stderr_lines": ["time="2022-12-06T22:38:47Z" level=fatal msg="failed to copy: httpReadSeeker: failed open: unexpected status code https://registry-1.docker.io/v2/library/nginx/manifests/sha256:4a846cc240449c53c8ae24269ba6bcaee5167d8ad75cd2a8d8ba422b7c726979: 429 Too Many Requests - Server message: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit""], "stdout": "", "stdout_lines": []}
The text was updated successfully, but these errors were encountered: