Support for oci:// scheme (Helm 3.5.0) #666
Comments
|
Ah, the |
|
Yes please, this is helpful for using Terraform with OCI registeries such as AWS ECR. |
aareet
added
needs-investigation
upstream-helm
|
Investigate when doing the work for #677 |
|
It looks like #677 has a PR up, but at a glance it doesn't look like this is addressed. Is it still on the roadmap? |
|
The PR to update Helm involves a lot of files getting update due to vendoring. I think it's better off to explore this in a separate PR once the dependency has been upgraded. |
|
Looking at the Helm code, it seems that the only thing to do - now that #709 has been merged - is to set the |
|
@wjam Unfortunately it doesn't seem to work for me. Needs more work to manually pull the chart first. |
|
As i suggest problem is here helm/helm#6982 upd: Here is my temp WA for this case: |
|
I have created a provider that can pull a helm oci registry chart to local chart by a data attribute. This way we don't have to use local resources until this is implemented in the helm provider :) https://registry.terraform.io/providers/SvenHamers/helmoci/latest |
|
is there anything blocking this? https://github.com/SvenHamers/terraform-provider-helmoci/blob/main/helmoci/data_oci.go looks to be whats missing, does it just need a pr? |
|
@ashtonian I used stuff from the internals of helm package. So if you want to do the same here you can't just use the helm package. Don't think it is the right way to do in a official provider |
|
@SvenHamers got it, thanks didn't realize it was still an issue upstream. |
|
Looks like helm/helm#9782 which will hopefully be merged before September gets closer and "the internal/experimental/registry package has been stabilized. The public-facing Login, Logout, Pull, and Push methods all follow a common pattern, taking options and returning a result object." although still marked as experimental. |
|
Hello All. With helm 3.7 being now released and containing support for OCI artifacts (still hidden behind |
|
there still seems to be a bug at least in the cli |
|
Helm 3.7 now supports this, AFAIK. My current workaround: resource "null_resource" "pull_my_chart" {
triggers = {
random = uuid()
}
provisioner "local-exec" {
command = <<-EOT
export HELM_EXPERIMENTAL_OCI=1
helm registry login registry.gitlab.com --username ${var.container_registry_username} --password ${var.container_registry_password}
helm pull oci://registry.gitlab.com/my-org/my-chart --version ${local.my_chart_version} --untar -d /tmp
EOT
}
}
resource "helm_release" "my_release" {
depends_on = [null_resource.pull_my_chart]
name = "my-release"
chart = "/tmp/my-chart"
version = local.my_chart_version
} |
|
I want to point out that there was a breaking change to the Helm OCI manifest in 3.7, so charts pushed with <3.7 may not work with >=3.7. I think there is little point in implementing support for the 3.5 format. |
Do you have any link to this issue? does it apply also the other way around? so manifests generated by Helm >= 3.7 not being supported by Helm <3.7 ? |
|
I believe that in newest 3.7.1 release the mediatype (different in helm version <3.7.0) was made backward compatible. So charts created by <3.7.0 should be installable with Helm 3.7.1. |
|
Hi Guys, is there anyone able to have a look at this issue? hope someone is able to fix it. |
|
Helm 3.7 changed its api. Now it can do Also, as zepellin mentioned. If charts were packaged pre 3.7, they need to get repackaged and pushed for it to work. But thats a limitation of helm itself. You have the same problem if you use helm manually. Its mentioned in the release notes https://github.com/helm/helm/releases/tag/v3.7.0. The below works for me with charts packaged by helm 3.7. Each chart has its own repo. In this example, emqx. resource "helm_release" "emqx_cluster" {
chart = "oci://${var.helm_chart_acr_fqdn}/helm/v2/emqx"
version = "0.3.2"
name = "emqx"
namespace = "mqtt"
create_namespace = true
atomic = true
values = var.emqx_values
} |
|
@bluebrown Thanks for the example. It works for me with Helm 3.7 as well. I repackaged the helm chart and pushed it to an Azure Container Registry. The following example uses an Azure Container Registry called helm-acr and the helm repository is called base-project. |
|
May I know where can I find the official documentation on how to use In particular it will be great if there are information on how to:
|
|
@chrissng Those infos are generally available in the helm docs. I don't know, but my guess is Terraform will only officially support this if it's out of experimental stage. Although, it's already supported just because helm install works now like that, as you see from the above answer. Here is an example of how to enable the experimental feature and login to ACR. # enable the experimental feature
export HELM_EXPERIMENTAL_OCI=1
# fetch own credentials and login with helm
az acr login \
--name myregistry \
--expose-token \
--output tsv \
--query accessToken \
2>/dev/null \
| helm registry login myregistry.azurecr.io \
--username 00000000-0000-0000-0000-000000000000 \
--password-stdin
# or use a service principal
helm registry login myregistry.azurecr.io -u <acr_sp_id> -p <acr_sp_pwd>
# perform some operation against the registry
helm show values oci://myregistry.azurecr.io/helm/v2/emqx --version 0.3.2 |
|
@bluebrown is that example for a public ACR repository? I'm trying with a private helm chart ECR registry but doesn't seem to be working for me. Update: Never mind, it was due to older version of the helm provider, tried with the |
|
Did you need to pass any creds into the helm provider to log into that registry? |
|
@jodybro, no you need to login helm manually outside of Terraform. Or you run those commands in something like null resource local exec. |
|
Ahh gotcha. Thanks! |
|
Anyone had success using data "aws_ecr_authorization_token" "token" {
registry_id = "0000000000000"
}
resource "helm_release" "chart_release" {
chart = "oci://${trimprefix(data.aws_ecr_authorization_token.token.proxy_endpoint,"https://")}/chart-name"
repository_username = data.aws_ecr_authorization_token.token.user_name
repository_password = data.aws_ecr_authorization_token.token.password
version = "0.2.0"
name = "chart-name"
} |
|
Another way this can be done is by manually writing out the |
|
In case it helps anyone, got this to pull a chart from github packages with provider variable "github_username" {
type = string
sensitive = true
}
variable "github_token" {
type = string
sensitive = true
}
resource "null_resource" "helm_login" {
triggers = {
always_run = timestamp()
}
provisioner "local-exec" {
command = <<-EOT
HELM_EXPERIMENTAL_OCI=1 \
helm registry login \
-u ${var.github_username} \
-p ${var.github_token} \
https://ghcr.io
EOT
}
}
resource "helm_release" "release" {
repository_username = var.github_username
repository_password = var.github_token
name = local.name
chart = local.chart # oci://ghcr.io/foo/bar/...
version = local.version
namespace = local.namespace
create_namespace = false
cleanup_on_fail = true
depends_on = [null_resource.helm_login]
}
|
|
Although using helm 3.7 cli work fine to pull and push to ECR, with terraform i cannot make it. I use the last helm provider, which normally use helm 3.7 but cannot make it work. The error is always the same Anyone succeeded to make it work ? PS : i exported the HELM_EXPERIMENTAL_OCI, i also tried login in with helm cli etc, but always the same error |
|
@Kent1 Did you re-package your Helm charts? See https://github.com/helm/helm/releases/tag/v3.7.0
I had a similar tf error before repackaging the charts. |
|
i did yes :( EDIT: Actually creating a new version worked .. I don't get it.. It was a brand new empty repo and i repackaged the helm chart juste before pushing. i'll do some more test but thanks @schra |
|
I'm having issues with this as well but I packaged and deployed the Helm chart using version 3.8.0 of the CLI. I get the error |
@rpf3 I'm on github actions, and I had to force downgrade the host to 3.7.2 to get it to work (I think what's happening is the terraform resource uses the 3.7.X version of the golang lib, but the latest helm CLI 3.8.X backwards broke the OCI creds, still need to verify this is the cause) here are my two github action steps before I apply terraform: |
|
@zenyui I think you may be correct. I just downgraded my local Helm client and was able to pull the chart from ECR successfully using the |
|
@rpf3, would you have a working snippet using I ended up creating a similar workaround to the ones proposed in the discussion, but would love to have a "cleaner" solution for this issue. By the way, in case it is helpful, here's the workaround: |
|
@vmendiolalau here's the outline of my code. I guess maybe my comment was unclear but I'm using the authorization token as the credentials in my #
# Helm Authentication
#
locals {
registry_uri = format("%s.dkr.ecr.%s.amazonaws.com", var.account_id, var.primary_region)
}
data "aws_ecr_authorization_token" "token" {
registry_id = var.account_id
}
resource "null_resource" "helm_login" {
triggers = {
always_run = timestamp()
}
provisioner "local-exec" {
command = <<-EOT
HELM_EXPERIMENTAL_OCI=1 \
helm registry login \
--username "${data.aws_ecr_authorization_token.token.user_name}" \
--password "${data.aws_ecr_authorization_token.token.password}" \
"${local.registry_uri}"
EOT
}
}
#
# Helm Release
#
resource "helm_release" "my_release" {
name = "my-app"
chart = format("oci://%s/my-repo-name", local.registry_uri)
version = "my-tag"
namespace = kubernetes_namespace.my_namespace.metadata[0].name
depends_on = [
null_resource.helm_login
]
} |
|
Has anyone gotten this to work on gcp with artifact registry? I keep getting resource "null_resource" "ouath_repo" {
triggers = {
random = uuid()
}
provisioner "local-exec" {
command = <<-EOT
export HELM_EXPERIMENTAL_OCI=1
gcloud auth print-access-token --project=myproject | helm registry login -u oauth2accesstoken --password-stdin https://europe-west2-docker.pkg.dev
EOT
}
}
resource "helm_release" "springapp" {
//depends_on = [null_resource.ouath_repo]
name = "springapp"
version = "0.0.0-feature-wif-SNAPSHOT"
//repository ="https://europe-west2-docker.pkg.dev/myproject/docker-helm"
create_namespace = true
namespace = "default"
wait = false
verify = false
chart = "oci://europe-west2-docker.pkg.dev/myproject/docker-helm/springapp"
values = [
//file("${path.module}/values.yml")
]
set {
name = "spring.profiles.active"
value = var.project
}
}
provider "helm" {
debug = true
kubernetes {
host = "https://${data.google_container_cluster.cluster.endpoint}"
token = data.google_client_config.provider.access_token
cluster_ca_certificate = base64decode(
data.google_container_cluster.cluster.master_auth[0].cluster_ca_certificate,
)
}
} |
|
Hopefully, this will be resolved when #837 is merged |
I built that branch and overrode the provider, but now I'm getting a new error: And here's my terraform manifest: Edit: I'm not super familiar with how helm handles OCI repositories, but it looks like the error is coming from here: https://github.com/helm/helm/blob/main/pkg/action/install.go#L679. Though I'm not sure why the client isn't defined. I can run |
|
Thanks for trying out the above branch @LP0101 – there were some changes I had to make to set up the OCI registry client in the provider, and run the login operation if the username/password were specified. It should be working now if you want to try again with the latest push to the branch. |
|
@jrhouston Looks like it works with OCI charts now. That was quick, thanks a lot! |
|
OCI support should now be available in the latest release. Thanks for your patience all 😄. |
This patch allows the use of ACR as a repository for helm charts.
Example usage:
helm_charts = {
mychart = {
name = "mychart"
chart = "mychart"
namespace = "default"
version = "0.0.1"
azure_container_registry = {
lz_key = "devops"
key = "devops_acr"
username = "00000000-0000-0000-0000-000000000000"
}
}
}
Note, the version bump of the helm provider is for the recently added
oci support: hashicorp/terraform-provider-helm#666
See https://github.com/hashicorp/terraform-provider-helm/releases/tag/v2.5.0 and hashicorp/terraform-provider-helm#666 Signed-off-by: Scott Rigby <scott@r6by.com>
|
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. |
and others
Description
Helm 3.5.0 now supports
oci://scheme for pulling dependencies from OCI registries. Can we upgrade to it?Potential Terraform Configuration
Community Note
The text was updated successfully, but these errors were encountered: