namespace not supported by helmCharts #3815

james-callahan · 2021-04-19T00:37:53Z

The new helmCharts feature (added in #3784) has no way to pass in a namespace.
This is problematic with many common charts where the namespace is used in e.g. arguments or annotations

e.g. try rendering the cert-manager helm chart:

helmCharts:
- name: cert-manager
  releaseName: cert-manager
  version: 1.3.0
  repo: https://charts.jetstack.io

Actual output

it results in annotations that contain the namespace, e.g.
cert-manager.io/inject-ca-from-secret: default/cert-manager-webhook-ca

it also has the argument:
- --dynamic-serving-dns-names=cert-manager-webhook,cert-manager-webhook.default,cert-manager-webhook.default.svc

Kustomize version/Platform

{Version:4.1.2 GitCommit:$Format:%H$ BuildDate:2021-04-19T00:04:53Z GoOs:linux GoArch:amd64}

CC @monopole

The text was updated successfully, but these errors were encountered:

Shell32-Natsu · 2021-04-19T16:59:21Z

#3816 (comment)

monopole · 2021-04-23T17:51:24Z

The helm program does many many things, including (so I'm told) arbitrary execution of shell commands via --extra-args and particularly crafted Go templates.

The basic chart inflation kustomize performs is intentionally rudimentary to prove out the concept of performing "last mile" chart customization. Maintaining kustomize long term as a helm runner with full helm flag and argument knowledge as helm evolves would be toilsome and imply kustomize inherits whatever security risks helm presents.

So - anything that looks like what kustomize does should be done by kustomize, rather than delegated to helm, e.g. setting namespace (#3838).

The safest, most performant, and completely unlimited way to use kustomize + helm to generate config is

manually inflate the helm chart via helm template --foo --bar ... (with whatever risky options one wants),
commit the resulting plain old k8s YAML to a repo.
use kustomize to operate on that repo, treating it as a base, adding some last mile config changes.

From time to time, capture the latest chart, and repeat steps 1 and 2. It's dangerous to set up a production stack that relies on kustomize to freshly download a helm chart with each build.

monopole · 2021-04-23T19:05:29Z

BTW, i'm not asserting that #3838 yields the result you want.

I'm wondering what the full use case is, and if it can be satisfied in the fashion suggested by #3838

cehoffman · 2021-04-23T20:05:15Z

So - anything that looks like what kustomize does should be done by kustomize, rather than delegated to helm, e.g. setting namespace (#3838).

I think the namespace setting in kustomize should flow into the namespace setting of helm, however I didn't see a way to get that value in the helm inflator plugin. The namespace setting of kustomize currently can not satisfy what the namespace setting of helm does, e.g. setting the namespace component in the middle of an annotation string to the resource like cert-manager uses.

From time to time, capture the latest chart, and repeat steps 1 and 2. It's dangerous to set up a production stack that relies on kustomize to freshly download a helm chart with each build.

Totally agree. Ideally you don't do that, but it comes down to scheduling prioritization and resources to implement. For me, submitting the PR was less resources to get kustomize in the door than setting up the rest before I could even start with kustomize and our many other usecases that don't usere helm where kustomize would help me.

monopole · 2021-04-23T22:59:03Z

I think the namespace setting in kustomize should flow into the namespace setting of helm.

That's interesting, and would avoid the easy contradictions one can imagine by not doing it that way.

drivelikebrazil · 2021-04-28T13:38:50Z

The helm program does many many things, including (so I'm told) arbitrary execution of shell commands via --extra-args and particularly crafted Go templates.

This is an issue with Helm's --post-renderer option in particular and the scary part was that someone running kustomize didn't have to consciously opt-in to run Helm. Given that we now have to explicitly elect to run Helm when we run kustomize, the user is consciously choosing to run Helm and accept the risks associated with it.

I think the namespace setting in kustomize should flow into the namespace setting of helm

This is a partially workable solution to me. The main places I could see it causing issues is with charts like Jenkins that create resources in other namespaces intentionally (in this case, creating a role in the agent namepace to allow Jenkins to spin up agent pods there).

From time to time, capture the latest chart, and repeat steps 1 and 2. It's dangerous to set up a production stack that relies on kustomize to freshly download a helm chart with each build.

Perhaps if you're pulling the Helm chart from an uncontrolled source. We pull specific versions of Helm charts into a private Helm repo and enforce tag immutability from there. We deploy multiple instances of a given application with different configurations and storing the fully templated code for each deployment adds a ton of noise and places for user error.

monopole · 2021-05-11T03:50:25Z

This is a situation where one could use a transformer to change the configuration of another transformer before that generator/transformer is applied.

https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/#transformed-transformers

monopole · 2021-05-11T20:20:55Z

Reopening to track the broader concept of having only one place to specify a namespace change.
Right now, if you use helm charts, you have to set it there too.

rdxmb · 2021-06-21T10:04:53Z

I have tried to define the namespace twice. Works with https://github.com/grafana/helm-charts/tree/main/charts/grafana

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: grafana-helm-test
helmCharts:
- name: grafana
  releaseName: customer-metrics-ui-v1
  repo: https://grafana.github.io/helm-charts
  namespace: grafana-helm-test
  version:  6.13.0

$ kustomize version
{Version:kustomize/v4.1.3 GitCommit:0f614e92f72f1b938a9171b964d90b197ca8fb68 BuildDate:2021-05-20T20:52:40Z GoOs:linux GoArch:amd64}

k8s-triage-robot · 2021-09-19T10:25:01Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

qkflies · 2021-09-20T02:56:02Z

/remove-lifecycle stale

k8s-triage-robot · 2021-12-19T03:53:35Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

rcmorano · 2022-01-10T10:54:50Z

I have tried to define the namespace twice. Works with https://github.com/grafana/helm-charts/tree/main/charts/grafana

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: grafana-helm-test
helmCharts:
- name: grafana
  releaseName: customer-metrics-ui-v1
  repo: https://grafana.github.io/helm-charts
  namespace: grafana-helm-test
  version:  6.13.0

$ kustomize version
{Version:kustomize/v4.1.3 GitCommit:0f614e92f72f1b938a9171b964d90b197ca8fb68 BuildDate:2021-05-20T20:52:40Z GoOs:linux GoArch:amd64}

If I'm not wrong, this wouldn't work with many multi-namespace charts as it would try to use the main namespace for every resource. It doesn't for me with this [0] fission, ending up with this [1] error.

It would be great to make kustomize use the behaviour proposed by @cehoffman; that's what I would expect instead of any fancy transformation :D

[0]

namespace: fission

bases:
- https://github.com/fission/fission/crds/v1?ref=v1.15.1

helmCharts:
- name: fission-all
  valuesFile: helm-values.yaml
  namespace: fission
  releaseName: init0
  version: v1.15.1
  repo: https://fission.github.io/fission-charts/

[1]

E0110 11:50:44.766222 4050399 run.go:120] "command failed" err="namespace transformation produces ID conflict: [{\"apiVersion\":\"v1\",\"kind\":\"Namespace\",\"metadata\":{\"annotations\":{\"internal.config.kubernetes.io/previousKinds\":\"Namespace\",\"internal.config.kubernetes.io/previousNames\":\"fission-function\",\"internal.config.kubernetes.io/previousNamespaces\":\"_non_namespaceable_\"},\"labels\":{\"chart\":\"fission-all-v1.15.1\",\"name\":\"fission-function\"},\"name\":\"fission\"}} {\"apiVersion\":\"v1\",\"kind\":\"Namespace\",\"metadata\":{\"annotations\":{\"internal.config.kubernetes.io/previousKinds\":\"Namespace\",\"internal.config.kubernetes.io/previousNames\":\"fission-builder\",\"internal.config.kubernetes.io/previousNamespaces\":\"_non_namespaceable_\"},\"labels\":{\"chart\":\"fission-all-v1.15.1\",\"name\":\"fission-builder\"},\"name\":\"fission\"}}]"

k8s-triage-robot · 2022-04-10T11:48:00Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

rcmorano · 2022-04-23T07:55:20Z

bump

k8s-triage-robot · 2022-05-23T08:29:43Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

rcmorano · 2022-05-23T09:48:30Z

bump

k8s-triage-robot · 2022-06-22T10:36:46Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue or PR with /reopen
Mark this issue or PR as fresh with /remove-lifecycle rotten
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

k8s-ci-robot · 2022-06-22T10:36:55Z

@k8s-triage-robot: Closing this issue.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied

After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied

After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue or PR with /reopen

Mark this issue or PR as fresh with /remove-lifecycle rotten

Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

wibed · 2022-07-13T01:28:17Z

please reopen.

just to again underline an already made point.
it breaks applied charts for some are created in mind with those capabilities.
this must be taken into consideration, before electing "kustomize" as a tool to provision your nodes.

atleast i would.

wibed · 2022-07-13T01:38:39Z

/remove-lifecycle rotten

k8s-ci-robot · 2022-07-13T01:39:20Z

@wibed: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

wibed · 2022-07-13T01:41:20Z

could you or one of the contributors reopen this issue
@KnVerey

KnVerey · 2022-07-13T22:59:19Z

This issue was about the lack of a field that was added in #3832. #4593 implies it may have since been broken, but it's better to track that in that newer issue.

arielweinberger · 2022-08-26T01:27:04Z

Bumping again as this breaks our flow of specifying namespace in overlays completely.

marcjimz · 2023-04-20T00:43:34Z

The community position on some of these flaws are baffling.

I cannot inline edit multi-line string values for some of my YAML objects after flattening a helm chart, so I go towards the helmChart generator for an exception basis.

I cannot use the helmChart generator effectively, because it does not honor the namespace.

What are my remaining options?

wibed · 2023-10-29T06:13:11Z

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: XXXX

helmCharts:
...

sets the namespace correctly, if the chart makes use of the {.Release.Namespace} attribute

Skaronator · 2024-03-05T15:49:28Z

@wibed {{ .Release.Namespace }} doesn't fully work for me.

My kustomize file looks like this:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: the-bug

helmGlobals:
  chartHome: ../../_charts

helmCharts:
- name: namespace
  releaseName: namespace
  valuesFile: values.yaml

My Helm Chart looks like this:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: namespace-quota
  namespace: {{ .Release.Namespace }}
  annotations:
    release-namespace: {{ .Release.Namespace }}
spec: {{ .Values.resourceQuota | toYaml | nindent 2 }}

And building this looks like this:

$ kustomize build . --enable-helm
apiVersion: v1
kind: Namespace
metadata:
  labels:
    app.kubernetes.io/instance: namespace
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: default   #  <- {{ .Release.Namespace }} is still "default"
    helm.sh/chart: namespace-1.0.0
  name: the-bug
---
apiVersion: v1
kind: ResourceQuota
metadata:
  annotations:
    release-namespace: default        # <- {{ .Release.Namespace }} is still "default"
  name: namespace-quota
  namespace: the-bug                  # <- {{ .Release.Namespace }} correctly set
spec:
  hard:
    pods: 100

I suspect that kustomize just replaces the namespace values with the correct values.

This is an issue for us since we dynamically generate namespace and therefore cannot set namespace within a helmCharts block.

GenaSG · 2024-03-06T11:35:12Z

@wibed {{ .Release.Namespace }} doesn't fully work for me.

My kustomize file looks like this:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: the-bug

helmGlobals:
  chartHome: ../../_charts

helmCharts:
- name: namespace
  releaseName: namespace
  valuesFile: values.yaml

My Helm Chart looks like this:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: namespace-quota
  namespace: {{ .Release.Namespace }}
  annotations:
    release-namespace: {{ .Release.Namespace }}
spec: {{ .Values.resourceQuota | toYaml | nindent 2 }}

And building this looks like this:

$ kustomize build . --enable-helm
apiVersion: v1
kind: Namespace
metadata:
  labels:
    app.kubernetes.io/instance: namespace
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: default   #  <- {{ .Release.Namespace }} is still "default"
    helm.sh/chart: namespace-1.0.0
  name: the-bug
---
apiVersion: v1
kind: ResourceQuota
metadata:
  annotations:
    release-namespace: default        # <- {{ .Release.Namespace }} is still "default"
  name: namespace-quota
  namespace: the-bug                  # <- {{ .Release.Namespace }} correctly set
spec:
  hard:
    pods: 100

I suspect that kustomize just replaces the namespace values with the correct values.

This is an issue for us since we dynamically generate namespace and therefore cannot set namespace within a helmCharts block.

I'm having exactly the same issue. {{ .Release.Namespace }} is used in openmetadata-dependencies helm chart in multiple places to generate env vars and configs and default namespace used everywhere even though resources deployed to correct custom namespace.

Skaronator · 2024-03-07T10:53:38Z

I've retested with kustomize v5.3.0 since the included version in kubectl is quite old, but the issue persists.

wibed · 2024-03-07T17:27:02Z

I've retested with kustomize v5.3.0 since the included version in kubectl is quite old, but the issue persists.

i confirm that my workaround is more than flawed.

to set a overall namespace overrides internal set namespaces disregarding the variable {{ Release.Namespace }} entirely.

Skaronator · 2024-03-09T09:34:50Z

I created a new issue dedicated to this. You are welcome to comment and upvote #5566 :)

james-callahan added the kind/bug Categorizes issue or PR as related to a bug. label Apr 19, 2021

Shell32-Natsu added kind/feature Categorizes issue or PR as related to a new feature. and removed kind/bug Categorizes issue or PR as related to a bug. labels Apr 19, 2021

drivelikebrazil mentioned this issue Apr 20, 2021

helmCharts doesn't support --include-crds #3819

Closed

Shell32-Natsu added the triage/accepted Indicates an issue or PR is ready to be actively worked on. label Apr 20, 2021

qkflies mentioned this issue Apr 22, 2021

helmCharts doesn't support --skip-tests #3831

Closed

cehoffman mentioned this issue Apr 22, 2021

Add ability to set target namespace for helm chart template #3832

Merged

monopole mentioned this issue Apr 23, 2021

A last-mile helm example, with namespace and prefix #3838

Merged

monopole mentioned this issue Apr 23, 2021

helmCharts doesn't support APIVersions #3816

Closed

This comment has been minimized.

Sign in to view

k8s-ci-robot closed this as completed in #3832 May 11, 2021

monopole reopened this May 11, 2021

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 19, 2021

k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 20, 2021

tao12345666333 mentioned this issue Oct 26, 2021

Fix Helm Chart remove Capabilities.APIVersions for Kustomize to parse file kubernetes/ingress-nginx#7829

Merged

9 tasks

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Dec 19, 2021

natasha41575 removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 6, 2022

stephen-dexda mentioned this issue Feb 20, 2022

Using helmCharts doesn't work with charts that use .Release fields #4303

Closed

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 10, 2022

k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels May 23, 2022

jberger mentioned this issue Jun 9, 2022

HelmChartInflationGenerator ignores releaseNamespace #4593

Open

k8s-ci-robot closed this as completed Jun 22, 2022

wibed mentioned this issue Jul 13, 2022

helm inflator kustomize not same result as helm install cert-manager/cert-manager#5293

Closed

k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Jul 13, 2022

wibed mentioned this issue Aug 3, 2023

helmChart inflator friendly chart, including namespace metallb/metallb#2033

Closed

8 tasks

panteparak mentioned this issue Apr 11, 2024

Argocd with kustomize & helm doesn't replace all "{{ .Release.namespace }}" references argoproj/argo-cd#17803

Open

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

namespace not supported by helmCharts #3815

namespace not supported by helmCharts #3815

james-callahan commented Apr 19, 2021

Shell32-Natsu commented Apr 19, 2021

monopole commented Apr 23, 2021 •

edited

Loading

monopole commented Apr 23, 2021

cehoffman commented Apr 23, 2021 •

edited

Loading

monopole commented Apr 23, 2021

drivelikebrazil commented Apr 28, 2021

This comment has been minimized.

monopole commented May 11, 2021 •

edited

Loading

This comment has been minimized.

monopole commented May 11, 2021

rdxmb commented Jun 21, 2021 •

edited

Loading

k8s-triage-robot commented Sep 19, 2021

qkflies commented Sep 20, 2021

k8s-triage-robot commented Dec 19, 2021

rcmorano commented Jan 10, 2022 •

edited

Loading

k8s-triage-robot commented Apr 10, 2022

rcmorano commented Apr 23, 2022

k8s-triage-robot commented May 23, 2022

rcmorano commented May 23, 2022

k8s-triage-robot commented Jun 22, 2022

k8s-ci-robot commented Jun 22, 2022

wibed commented Jul 13, 2022

wibed commented Jul 13, 2022

k8s-ci-robot commented Jul 13, 2022

wibed commented Jul 13, 2022 •

edited

Loading

KnVerey commented Jul 13, 2022

arielweinberger commented Aug 26, 2022

marcjimz commented Apr 20, 2023

wibed commented Oct 29, 2023

Skaronator commented Mar 5, 2024

GenaSG commented Mar 6, 2024

Skaronator commented Mar 7, 2024

wibed commented Mar 7, 2024

Skaronator commented Mar 9, 2024

namespace not supported by helmCharts #3815

namespace not supported by helmCharts #3815

Comments

james-callahan commented Apr 19, 2021

Shell32-Natsu commented Apr 19, 2021

monopole commented Apr 23, 2021 • edited Loading

monopole commented Apr 23, 2021

cehoffman commented Apr 23, 2021 • edited Loading

monopole commented Apr 23, 2021

drivelikebrazil commented Apr 28, 2021

This comment has been minimized.

monopole commented May 11, 2021 • edited Loading

This comment has been minimized.

monopole commented May 11, 2021

rdxmb commented Jun 21, 2021 • edited Loading

k8s-triage-robot commented Sep 19, 2021

qkflies commented Sep 20, 2021

k8s-triage-robot commented Dec 19, 2021

rcmorano commented Jan 10, 2022 • edited Loading

k8s-triage-robot commented Apr 10, 2022

rcmorano commented Apr 23, 2022

k8s-triage-robot commented May 23, 2022

rcmorano commented May 23, 2022

k8s-triage-robot commented Jun 22, 2022

k8s-ci-robot commented Jun 22, 2022

wibed commented Jul 13, 2022

wibed commented Jul 13, 2022

k8s-ci-robot commented Jul 13, 2022

wibed commented Jul 13, 2022 • edited Loading

KnVerey commented Jul 13, 2022

arielweinberger commented Aug 26, 2022

marcjimz commented Apr 20, 2023

wibed commented Oct 29, 2023

Skaronator commented Mar 5, 2024

GenaSG commented Mar 6, 2024

Skaronator commented Mar 7, 2024

wibed commented Mar 7, 2024

Skaronator commented Mar 9, 2024

monopole commented Apr 23, 2021 •

edited

Loading

cehoffman commented Apr 23, 2021 •

edited

Loading

monopole commented May 11, 2021 •

edited

Loading

rdxmb commented Jun 21, 2021 •

edited

Loading

rcmorano commented Jan 10, 2022 •

edited

Loading

wibed commented Jul 13, 2022 •

edited

Loading