Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bitnami/multus-cni] The registry-1.docker.io registry isn't working. Update documentation to use the correct registry. #19599

Closed
roncemer opened this issue Sep 28, 2023 · 20 comments
Assignees
Labels
multus-cni solved tech-issues The user has a technical issue about an application triage Triage is needed

Comments

@roncemer
Copy link

roncemer commented Sep 28, 2023

Name and Version

bitnami/multus-cni 1.1.2

What architecture are you using?

amd64

What steps will reproduce the bug?

The registry-1.docker.io/bitnamicharts registry is throttling requests, preventing helm from installing the bitnami's multus-cni helm chart from there using this command:

helm upgrade --install multus-cni oci://registry-1.docker.io/bitnamicharts/multus-cni --version 1.1.2

Instead, you have to use these commands:

$ helm repo add bitnami https://charts.bitnami.com/bitnami
$ helm install my-multus-cni bitnami/multus-cni --version 1.1.2

The correct commands are shown in the INSTALL button on the right of this page: https://artifacthub.io/packages/helm/bitnami/multus-cni

Please update the documentation in the top-level README.md in this project accordingly.

What do you see instead?

Two different responses from registry-1.docker.io:

$ helm pull oci://registry-1.docker.io/bitnamicharts/multus-cni
Error: failed to authorize: failed to fetch anonymous token: unexpected status from GET request to https://auth.docker.io/token?scope=repository%3Abitnamicharts%2Fmultus-cni%3Apull&service=registry.docker.io: 401 Unauthorized

$ helm pull oci://registry-1.docker.io/bitnamicharts/multus-cni
Error: GET "https://registry-1.docker.io/v2/bitnamicharts/multus-cni/tags/list": unexpected status code 503: Service Unavailable
@roncemer roncemer added the tech-issues The user has a technical issue about an application label Sep 28, 2023
@github-actions github-actions bot added the triage Triage is needed label Sep 28, 2023
@carrodher
Copy link
Member

Yesterday for a few hours, there was an issue in DockerHub affecting registries, see https://www.dockerstatus.com/pages/history/533c6539221ae15e3f000031

At this moment everything is working as expected:

$ helm pull oci://registry-1.docker.io/bitnamicharts/multus-cni
Pulled: registry-1.docker.io/bitnamicharts/multus-cni:1.1.2
Digest: sha256:cb85154b4a4e5bac3d5690eaeffdde4534d327665f9eb258d884c2bd80a666b8

Although the bitnami repo and index.yaml are still available for backward compatibility, the default and recommended way of installing Bitnami Helm charts is using the OCI artifacts stored in DockerHub.

@kofemann
Copy link

kofemann commented Sep 29, 2023

I get the same issue with postgres

$ helm pull oci://registry-1.docker.io/bitnamicharts/postgres                                                 
Error: GET "https://registry-1.docker.io/v2/bitnamicharts/postgres/tags/list": unexpected status code 401: unauthorized: authentication required

However, I have no issues running postgres container manually

@david-sykora
Copy link

I observe the same behavior for several different charts. In some locations they work, in others I'm gettting the error. Looks like today is a busy day for cloud in general, multiple services are behaving strangely :)

@roncemer
Copy link
Author

Would it be possible to add the bitnami registry to the README.md as a fallback, so people can use that if the "official" registry is broken?

@Jbaukens
Copy link

Jbaukens commented Oct 1, 2023

Hello, It looks like we are getting the same issue somehow since a few days. Has someone been able to workaround this ?

Error: could not download oci://registry-1.docker.io/bitnamicharts/common: failed to authorize: failed to fetch anonymous token: unexpected status from GET request to https://auth.docker.io/token?scope=repository%3Abitnamicharts%2Fcommon%3Apull&service=registry.docker.io: 401 Unauthorized
helm.go:84: [debug] failed to authorize: failed to fetch anonymous token: unexpected status from GET request to https://auth.docker.io/token?scope=repository%3Abitnamicharts%2Fcommon%3Apull&service=registry.docker.io: 401 Unauthorized
could not download oci://registry-1.docker.io/bitnamicharts/common
helm.sh/helm/v3/pkg/downloader.(*Manager).downloadAll
	helm.sh/helm/v3/pkg/downloader/manager.go:356
helm.sh/helm/v3/pkg/downloader.(*Manager).Update
	helm.sh/helm/v3/pkg/downloader/manager.go:200
main.newDependencyUpdateCmd.func1
	helm.sh/helm/v3/cmd/helm/dependency_update.go:74
github.com/spf13/cobra.(*Command).execute
	github.com/spf13/cobra@v1.7.0/command.go:940
github.com/spf13/cobra.(*Command).ExecuteC
	github.com/spf13/cobra@v1.7.0/command.go:1068
github.com/spf13/cobra.(*Command).Execute
	github.com/spf13/cobra@v1.7.0/command.go:992
main.main
	helm.sh/helm/v3/cmd/helm/helm.go:83
runtime.main
	runtime/proc.go:250
runtime.goexit
	runtime/asm_amd64.s:1598

@ASCE1885
Copy link

ASCE1885 commented Oct 1, 2023

Error: INSTALLATION FAILED: failed to authorize: failed to fetch anonymous token: unexpected status from GET request to https://auth.docker.io/token?scope=repository%3Aapache%2Fskywalking-helm%3Apull&service=registry.docker.io: 401 Unauthorized

the same issue with skywalking and kafka

@david-sykora
Copy link

As mentioned above, it is possible to use directly the helm repository from Bitnami. You can search for desired chart on https://artifacthub.io/. For example a Kafka Helm chart would be used like this:

# add repository
helm repo add bitnami https://charts.bitnami.com/bitnami
# install chart
helm install my-kafka bitnami/kafka --version 25.2.0

@javsalgar javsalgar changed the title The registry-1.docker.io registry isn't working. Update documentation to use the correct registry. [bitnami/multus-cni] The registry-1.docker.io registry isn't working. Update documentation to use the correct registry. Oct 2, 2023
@rblaine95
Copy link
Contributor

Although the bitnami repo and index.yaml are still available for backward compatibility [...]

Does this mean that https://charts.bitnami.com/bitnami will be removed at some point in the future?

@carrodher
Copy link
Member

Although the bitnami repo and index.yaml are still available for backward compatibility [...]

Does this mean that https://charts.bitnami.com/bitnami will be removed at some point in the future?

It may undergo changes compared to how it works today, yes. It is not something that is planned in the short/medium term so it is not known what is the nature of those potential changes; but given that the DockerHub OCI registry is the default and preferred way to deploy Bitnami Helm charts, any other method it is susceptible to change in the future.

@muhlba91
Copy link

muhlba91 commented Oct 5, 2023

for me this affects all charts from bitnami i'm using and is preventing me from upgrading these charts on the clusters, or rolling them out to new clusters.

@carrodher imo, if using the OCI chart is the recommended and default way to install bitnami charts it should work.
i do understand that https://charts.bitnami.com/bitnami is still functioning but i'd appreciate clear communication if this OCI issue is being addressed (soon) or if using https://charts.bitnami.com/bitnami is the recommended and preferred resolution now.

@carrodher
Copy link
Member

I understand that you're facing some issues with the OCI registry, and I'm here to help.

It's important to note that the OCI registry isn't hosted or maintained by our team. If you're encountering problems, I recommend reaching out to the DockerHub community forums. Helm charts, just like container images for several years, are hosted on an external trusted registry like DockerHub, although it's true that during the latest days, they faced some issues. That's why my recommendation is to report there any issues.

Regarding https://charts.bitnami.com/bitnami, I want to clarify that although it's currently available, it's not the recommended way to consume Bitnami Helm charts. While we have no immediate plans to decommission it, we'll provide ample notice to users if any changes are on the horizon. If such a decision is ever made, we'll ensure a proper communication strategy is in place to give users sufficient time to adapt.

If you have any more questions or need additional information, please don't hesitate to ask.

@flypenguin
Copy link

i lost literally hours now until i found this thread. the oci registry seems unstable as hell and in no way fit for production use. please keep "the old way" around for the time being, and document it prominently, because this is time burnt.

@iaacautomation
Copy link

Dockerhub is broken for oci registry, its basically useless, everytime auth errors, there is no option to pull without auth.

@seancallaway
Copy link

Having the same issue with Redis today. I see @carrodher's comment and agree, but should moving the "recommended way" to something that works and is more stable?

@javsalgar
Copy link
Contributor

FWIW, I created a post in Docker forums https://forums.docker.com/t/docker-hub-oci-registry-for-helm-charts-is-unstable-and-returns-401-errors/138043

@peikk0
Copy link

peikk0 commented Oct 11, 2023

I believe this is related to helm/helm#12423 which is fixed with helm/helm#12424 and should be released in Helm v3.13.1. I don't have this issue with Helm v3.12.

@5n00p4eg
Copy link

I agree that it's not a server-side problem. My local helm client works fine (v3.11) while CI has v3.13.0 and it gives me the error all the time.

@5n00p4eg
Copy link

Just checked the new helm version v3.13.1 and it fixes the issue for me.

@javsalgar
Copy link
Contributor

Thanks for letting us know

@jaihind213
Copy link

jaihind213 commented Apr 5, 2024

u need to add 'imagePullSecrets' to the spec.

create a file ' image-pull-secrets.yaml ' with content:

imagePullSecrets:
  - name: <secretName>

helm install blah blah blah -f image-pull-secrets.yaml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
multus-cni solved tech-issues The user has a technical issue about an application triage Triage is needed
Projects
None yet
Development

No branches or pull requests