failed to download kic base image or any fallback image (unable to access gcr.io) #8997
Comments
tstromberg
changed the title
|
Are you in China by any chance? If so, can you provide the output of @medyagh - is that description of the fallback behavior accurate? |
tstromberg
added
cause/firewall-or-proxy
how to fall-back to Github? |
|
Our code should automatically fallback to github once the call to GCR fails, which is why we wanted to see more detailed output from minikube start. Try running |
sharifelgamal
added
priority/awaiting-more-evidence
|
Hi, same problem: `` stderr: stderr: stderr: stderr: W0929 14:28:53.572546 6495 start.go:240] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 1 stderr: stderr: stderr: stderr: 🤦 StartHost failed, but will try again: creating host: create: creating: setting up container node: preparing volume for minikube container: docker run --rm --entrypoint /usr/bin/test -v minikube:/var gcr.io/k8s-minikube/kicbase:v0.0.12-snapshot3@sha256:1d687ba53e19dbe5fafe4cc18aa07f269ecc4b7b622f2251b5bf569ddb474e9b -d /var/lib: exit status 125 stderr: I0929 14:28:53.683266 6495 start.go:392] Will try again in 5 seconds ... stderr: stderr: stderr: stderr: stderr: stderr: stderr: stderr: stderr: stderr: stderr: stderr: stderr: stderr: stderr: stderr: stderr: stderr: stderr: stderr: stderr: stderr: stderr: stderr: I0929 14:29:21.889309 6495 cli_runner.go:110] Run: docker rm -f -v minikube stderr: stderr: stderr: stderr: stderr: W0929 14:29:45.658544 6495 start.go:240] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 1 stderr: stderr: stderr: stderr: stderr: stderr: W0929 14:29:48.204191 6495 start.go:240] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 1 stderr: stderr: 😿 Failed to start docker container. Running "minikube delete" may fix it: recreate: creating host: create: creating: setting up container node: preparing volume for minikube container: docker run --rm --entrypoint /usr/bin/test -v minikube:/var gcr.io/k8s-minikube/kicbase:v0.0.12-snapshot3@sha256:1d687ba53e19dbe5fafe4cc18aa07f269ecc4b7b622f2251b5bf569ddb474e9b -d /var/lib: exit status 125 stderr: I0929 14:29:48.225333 6495 out.go:109] W0929 14:29:48.227573 6495 out.go:145] ❌ Exiting due to GUEST_PROVISION: Failed to start host: recreate: creating host: create: creating: setting up container node: preparing volume for minikube container: docker run --rm --entrypoint /usr/bin/test -v minikube:/var gcr.io/k8s-minikube/kicbase:v0.0.12-snapshot3@sha256:1d687ba53e19dbe5fafe4cc18aa07f269ecc4b7b622f2251b5bf569ddb474e9b -d /var/lib: exit status 125 stderr: ❌ Exiting due to GUEST_PROVISION: Failed to start host: recreate: creating host: create: creating: setting up container node: preparing volume for minikube container: docker run --rm --entrypoint /usr/bin/test -v minikube:/var gcr.io/k8s-minikube/kicbase:v0.0.12-snapshot3@sha256:1d687ba53e19dbe5fafe4cc18aa07f269ecc4b7b622f2251b5bf569ddb474e9b -d /var/lib: exit status 125 stderr: W0929 14:29:48.233977 6495 out.go:145] W0929 14:29:48.234037 6495 out.go:145] 😿 If the above advice does not help, please let us know: |
|
same problem |
|
same issue |
|
If you are in China , you can |
|
I have the same problem |
Doesn't work |
|
Work!!! (base) ➜ ~ minikube start --image-mirror-country='cn' --image-repository='registry.cn-hangzhou.aliyuncs.com/google_containers' |
|
@baymax55 Can you expand on what's not working, are you still getting the same error as before or are you getting a new error now? |
|
I'm closing this issue as this issue should be fixed in recent releases of minikube ( v1.17+). If this issue does continue to exist in the most recent release of minikube, please feel free to re-open it by replying If someone sees a similar issue to this one, please re-open it as replies to closed issues are unlikely to be noticed. Thank you for opening the issue! |
|
/reopen |
|
@vonbrand: You can't reopen an issue/PR unless you authored it or you are a collaborator. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
@vonbrand what error are you getting exactly? |
spowelljr
added
long-term-support
|
❯ minikube start stderr: 🤷 docker "minikube" container is missing, will recreate. stderr: ❌ Exiting due to GUEST_PROVISION: Failed to start host: recreate: creating host: create: creating: setting up container node: preparing volume for minikube container: docker run --rm --name minikube-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --entrypoint /usr/bin/test -v minikube:/var kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e -d /var/lib: exit status 125 stderr: ╭────────────────────────────────────────────────────────────────────╮ ~/workspace/Ausbildung on chris-ryu/issue161 *20 ?1 |
|
I get the same thing with podman+minikube: |
when I use the latest version for minikube, there is no such problem, thanks |
|
Based on the error message posted here, it looks like there's a strange caching bug for kicbase when restarting an existing cluster. Can @bhundven or @chris-ryu confirm that this is still happening on minikube 1.22.0? For fresh starts, it looks like the fallback mechanism is working as intended. |
In my case, I set the |
|
(base) minikube start --driver=podman stderr: 🔄 Restarting existing podman container for "minikube" ... stderr: ❌ Exiting due to GUEST_PROVISION: Failed to start host: podman inspect ip minikube: sudo -n podman container inspect -f minikube: exit status 125 stderr: ╭───────────────────────────────────────────────────────────────────────────────────────────╮ It works for me！！！！ (base) minikube start --driver=podman --image-mirror-country='cn' --image-repository='registry.cn-hangzhou.aliyuncs.com/google_containers' |
|
im not in china and I get the same issue... running Mac OS 12.1 on apple (M1) silicon |
|
On Mac OS X 12.1., with an Intel CPU, it is the same. I got from the script the hint to delete the minikube image after it tried a different IP. I tried, and the output was the following: I switched off all firewall functionality before I tried podman vs. docker vs. footloose etc. I had issues with Footloose related to systemd-v2 in docker-desktop 4.3; not sure if that relates in any way? From all options I am testing, just docker works for the moment. |
|
I try the following command in my WSL2, and it can work $ minikube start --network-plugin=cni --cni=calico --driver=docker --base-image "gcr.io/k8s-minikube/kicbase-builds:v0.0.29-1644344181-13531" 😄 minikube v1.25.1 on Ubuntu 20.04 NOTE: The base image string I got from https://minikube.sigs.k8s.io/docs/commands/start/ |
|
same here, running in WSL2. It works after I set the manual proxy configuration in docker desktop. |
|
use |
|
If you are in China , try run: |
|
reconfiguration may fix the issue If you are not in cn
|
I use mac, it doesn't work for me |
|
rm .minikube folder |
|
Hello, I have tried My OS is Linux, Manjaro It looks like some of layers failed to download Any help is appreciated, thank you! Here is full log: |
|
Just ran into this same issue, I'm in Hawaii for the winter and was connected via a Spectrum v6 address. Switched to my mobile hotspot, an ATT v6 address and everything just worked. I have had some other sites seem to think I am connected from outside the US when using this spectrum connection here in Hawaii so perhaps there is some bad data trying to map the v6 address to a location? |
|
I am in china , I had the same problem until add --registry-mirror , now it work well，if you are in china , 阿里云镜像服务 may help you |
|
Same issue
|
1st update:It looks like the first command unnecessary. Because : the original answerThanks this answer, I solved eventually but a little different.Note that I failed twice before, the cuase maybe my proxy connection is not stable, because I am in China. I just set proxy in
My console log print: And this: In the end:I still have a question. Specifically, I was reading the official documentation for minikube and came across the section about proxy. The document mentions that "If a HTTP proxy is required to access the internet, you may need to pass the proxy connection information to both minikube and Docker using environment variables" However, I'm not sure about how to go about passing the proxy connection information to Docker.. Is it like this: |
|
Do you find new image in China now? |
|
I had this problem. The k8s related images were downloaded through other methods. |
|
继续跟帖，后来人注意！！！ $ minikube start --force \
> --kubernetes-version=v1.25.14 \
> --image-mirror-country=cn \
> --image-repository='registry.cn-hangzhou.aliyuncs.com/google_containers'
* minikube v1.31.2 on Centos 7.9.2009
- KUBECONFIG=/etc/kubernetes/admin.conf
! minikube skips various validations when --force is supplied; this may lead to unexpected behavior
* Automatically selected the docker driver. Other choices: none, ssh
* The "docker" driver should not be used with root privileges. If you wish to continue as root, use --force.
* If you are running minikube within a VM, consider using --driver=none:
* https://minikube.sigs.k8s.io/docs/reference/drivers/none/
X The requested memory allocation of 1963MiB does not leave room for system overhead (total system memory: 1963MiB). You may face stability issues.
* Suggestion: Start minikube with less memory allocated: 'minikube start --memory=1963mb'
* Using image repository registry.cn-hangzhou.aliyuncs.com/google_containers
* Using Docker driver with root privileges
* Starting control plane node minikube in cluster minikube
* Pulling base image ...
! minikube was unable to download registry.cn-hangzhou.aliyuncs.com/google_containers/kicbase:v0.0.40, but successfully downloaded docker.io/kicbase/stable:v0.0.40 as a fallback image
* Creating docker container (CPUs=2, Memory=1963MB) ...
* Preparing Kubernetes v1.25.14 on Docker 24.0.4 ...
X Exiting due to K8S_INSTALL_FAILED: Failed to update cluster: updating control plane: downloading binaries: downloading kubectl: download failed: https://kubernetes.oss-cn-hangzhou.aliyuncs.com/kubernetes-release/release/v1.25.14/bin/linux/amd64/kubectl?checksum=file:https://kubernetes.oss-cn-hangzhou.aliyuncs.com/kubernetes-release/release/v1.25.14/bin/linux/amd64/kubectl.sha256: getter: &{Ctx:context.Background Src:https://kubernetes.oss-cn-hangzhou.aliyuncs.com/kubernetes-release/release/v1.25.14/bin/linux/amd64/kubectl?checksum=file:https://kubernetes.oss-cn-hangzhou.aliyuncs.com/kubernetes-release/release/v1.25.14/bin/linux/amd64/kubectl.sha256 Dst:/root/.minikube/cache/linux/amd64/v1.25.14/kubectl.download Pwd: Mode:2 Umask:---------- Detectors:[0x3f9c8a8 0x3f9c8a8 0x3f9c8a8 0x3f9c8a8 0x3f9c8a8 0x3f9c8a8 0x3f9c8a8] Decompressors:map[bz2:0xc000431238 gz:0xc000431290 tar:0xc000431240 tar.bz2:0xc000431250 tar.gz:0xc000431260 tar.xz:0xc000431270 tar.zst:0xc000431280 tbz2:0xc000431250 tgz:0xc000431260 txz:0xc000431270 tzst:0xc000431280 xz:0xc000431298 zip:0xc0004312a0 zst:0xc0004312b0] Getters:map[file:0xc00107fea0 http:0xc001098500 https:0xc001098550] Dir:false ProgressListener:0x3f579a0 Insecure:false DisableSymlinks:false Options:[0x12d0880]}: invalid checksum: Error downloading checksum file: bad response code: 404
|
|
Same here: 🤔 ! minikube was unable to download gcr.io/k8s-minikube/kicbase:v0.0.42, but successfully downloaded docker.io/kicbase/stable:v0.0.42 as a fallback image Logs: I0106 02:04:24.278493 82771 out.go:296] Setting OutFile to fd 1 ...
stderr: stderr: stderr: |
|
The Kubernetes project currently lacks enough contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale |
k8s-ci-robot
added
the
lifecycle/stale
|
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten |
k8s-ci-robot
added
lifecycle/rotten
|
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /close not-planned |
|
@k8s-triage-robot: Closing this issue, marking it as "Not Planned". In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
and others
~ minikube start
Pulling base image ...
E0814 15:09:05.833724 5268 cache.go:175] Error downloading kic artifacts: failed to download kic base image or any fallback image
🔥 Creating docker container (CPUs=2, Memory=6144MB) ...
🤦 StartHost failed, but will try again: creating host: create: creating: setting up container node: preparing volume for minikube container: docker run --rm --entrypoint /usr/bin/test -v minikube:/var gcr.io/k8s-minikube/kicbase:v0.0.11@sha256:6fee59db7d67ed8ae6835e4bcb02f32056dc95f11cb369c51e352b62dd198aa0 -d /var/lib: exit status 125
~ docker pull gcr.io/k8s-minikube/kicbase:v0.0.11
Error response from daemon: Get https://gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
i can pull other images, just this failed
The text was updated successfully, but these errors were encountered: