Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kind cluster not able to pull public images from ghcr.io #3663

Open
rohitagg2020 opened this issue Jun 18, 2024 · 8 comments
Open

Kind cluster not able to pull public images from ghcr.io #3663

rohitagg2020 opened this issue Jun 18, 2024 · 8 comments
Labels
kind/support Categorizes issue or PR as a support question.

Comments

@rohitagg2020
Copy link

What happened:

I have created a kind cluster. But when I try to create a deployment (which pulls images from ghcr.io), it fails.

Error:

^ Pending: ErrImagePull (message: failed to pull and unpack image "ghcr.io/carvel-dev/kapp-controller@sha256:b83bd139c42777bb4428c2ae421f93675a7d6147dff3147471d1be112d9c1eec": 
failed to resolve reference "ghcr.io/carvel-dev/kapp-controller@sha256:b83bd139c42777bb4428c2ae421f93675a7d6147dff3147471d1be112d9c1eec": 
failed to do request: Head "https://ghcr.io/v2/carvel-dev/kapp-controller/manifests/sha256:b83bd139c42777bb4428c2ae421f93675a7d6147dff3147471d1be112d9c1eec": 
tls: failed to verify certificate: x509: certificate signed by unknown authority)

It has been working fine until last month. I thnk I have messed up my environment, because of which I am facing this issue.

Things I have tried:

  • Uninstall and install kind.
  • Deleted and created new kind cluster.
  • Uninstalled and reinstalled docker desktop.

But still, I am not able to get around the error. How can I get back to things working again?

What you expected to happen:
Deployment to be created successfully.
How to reproduce it (as minimally and precisely as possible):
kubectl apply -f https://github.com/carvel-dev/kapp-controller/releases/latest/download/release.yml
Anything else we need to know?:
As this is a public image, I guess the ca certs are not being picked up when kind cluster is created.
Environment:

  • kind version: (use kind version): kind v0.23.0
  • Runtime info: (use docker info, podman info or nerdctl info):
    Client: Docker Engine - Community
    Version: 26.1.4
    Context: desktop-linux
    Debug Mode: false
  • OS (e.g. from /etc/os-release): macOS
  • Kubernetes version: (use kubectl version):
    Client Version: v1.29.2
    Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
    Server Version: v1.27.9
  • Any proxies or other special environment settings?:
@rohitagg2020 rohitagg2020 added the kind/bug Categorizes issue or PR as related to a bug. label Jun 18, 2024
@stmcginnis
Copy link
Contributor

/remove-kind bug
/kind support

Please fill in the full "Environment" details from the issue template.

Are you in an environment that uses a proxy?

You could try deleting any cluster you have, then delete the kind Docker network, then try creating again. It will try to recreate the network to make sure there isn't something odd with the existing setup.

You can also do kind create cluster --retain, then use docker ps to see the created containers. Then you can docker exec into the kind container(s) to perform troubleshooting.

@k8s-ci-robot k8s-ci-robot added kind/support Categorizes issue or PR as a support question. and removed kind/bug Categorizes issue or PR as related to a bug. labels Jun 18, 2024
@BenTheElder
Copy link
Member

This looks like a MITM proxy causing issues.

@rohitagg2020
Copy link
Author

rohitagg2020 commented Jun 19, 2024

Are you in an environment that uses a proxy?

No

You could try deleting any cluster you have, then delete the kind Docker network, then try creating again. It will try to recreate the network to make sure there isn't something odd with the existing setup.

How can I do this?
I did try docker network remove but it didnt help.

@aojea
Copy link
Contributor

aojea commented Jun 19, 2024

docker network rm kind

https://docs.docker.com/reference/cli/docker/network/rm/

@rohitagg2020
Copy link
Author

I already tried docker network rm kind and it didnt work.

@tao12345666333
Copy link
Member

What's your steps to create kind cluster?

I saw your post server version is Server Version: v1.27.9

And kind version is 0.23

I think maybe you have changed some configurations

@rohitagg2020
Copy link
Author

What's your steps to create kind cluster?

kind create cluster

I think maybe you have changed some configurations

I guess so, but I dont remember it now. Is there any default path from which kind configuration is loaded? cc: @tao12345666333

@BenTheElder
Copy link
Member

no, you would've specified it https://kind.sigs.k8s.io/docs/user/configuration/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/support Categorizes issue or PR as a support question.
Projects
None yet
Development

No branches or pull requests

6 participants