Kustomize filter kinds on "base" input

[epcim]

Is your feature request related to a problem? Please describe.

There were already closed discussion whether Kustomize shall remove resources from it's "base" source and these IMO were closed as not a good practice.

I would like to reopen this or rather say more discussion/document what is "bases" for Kustomize. Sorry If I did missed some other issue with the same topic.

In the example bellow, you see Helm chart of longhorn provides this job, with annotation used to be executed on deployment removal:

  name: longhorn-uninstall
  annotations:
    helm.sh/hook: pre-delete

Why I would like to bring it to light again. This Helm chart in this case is "bad base" for the Kustomize. In fact not a direct bug of Kustomize itself, we could blame longhorn, but it's clear that some deployment annotations (not strictly Helm related) are being used for app life-cycle, but Kustomize itself will never take care of them.

Let's not speak about "filtering" resources only. We may want to add annotations, to mark, not to apply specific resource later. Obviously Kustomize is not yet in this business (not sure but kpt live applly - would treat that better?)

With KRM Fn available, with some more time with the project, how does the "Kustomize" team does feel what "bases" are and how they shall be used?

• Do we strictly expect "private" base (where one would do overrides for dev/prod etc)?
• Do we suppose, "base" code for application could be shared resource and Kustomize serve as swiss-knire, to overrides/patch on specific deployment?

If the 2nd is true, and many people use it this way from I have seen, then Helm or any KRM Fn can serve as generic manifest source. However without a primary control of such source (as it probably reside on some public repo, and you can only control it's global values.yaml etc..) one then miss the capability to pick only the Kinds wanted (or filter some not desired, not only transform them).

Imagine this source:

# cat ./longhorn/Kustomization 
# https://github.com/longhorn/charts/blob/master/charts/longhorn/values.yaml

namespace: longhorn-system

generatorOptions:
  disableNameSuffixHash: true

helmChartInflationGenerator:
- chartName: longhorn
  chartRepoUrl: https://charts.longhorn.io
  #chartVersion: 
  releaseName: longhorn
  releaseNamespace: longhorn-system
  valuesMerge: override
  extraArgs:
  - --include-crds
  valuesLocal:
    installCRDs: true
    fullnameOverride: longhorn

defaultSettings:
      defaultDataPath: "/var/lib/longhorn"

That will render, besides other manifests this, and guess (yes it's immutable if it exist so at first apply will fail, but on each apply you force it effectively do the task - uninstall whole. longhorn, including it's data.

---
apiVersion: batch/v1
kind: Job
metadata:
  annotations:
    helm.sh/hook: pre-delete
    helm.sh/hook-delete-policy: hook-succeeded
  labels:
    app.kubernetes.io/instance: longhorn
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: longhorn
    app.kubernetes.io/version: v1.2.3
    helm.sh/chart: longhorn-1.2.3
  name: longhorn-uninstall

Recent discussion, resources

• https://kubectl.docs.kubernetes.io/faq/kustomize/eschewedfeatures/#removal-directives

Describe the solution you'd like

• native way to filter kinds, based on name regex, labels and annotations
• update documentation in relation to what "bases" are considered, what Kustomize plays a role in "life-cycle" operations as shown in this example

Describe alternatives you've considered

• KRM Fn to filter Kinds would work, but seems like too complex and time consuming to run for this case.

[k8s-ci-robot]

@epcim: This issue is currently awaiting triage.

SIG CLI takes a lead on issue triage for this repo, but any Kubernetes member can accept issues by applying the triage/accepted label.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[KnVerey]

Do we strictly expect "private" base (where one would do overrides for dev/prod etc)?

Creating variants is indeed Kustomize's speciality, and we eschew features that would overcomplicate that basic use case. The doc you linked to does describe the intended workflow for when a base not under your control is not doing what you want:

If the underlying base is outside of one’s control, an OTS workflow is the recommended best practice. Fork the base, remove what you don’t want and commit it to your private fork, then use kustomize on your fork. As often as desired, use git rebase to capture improvements from the upstream base.

While I'm sympathetic to the desire not to maintain a fork, all of the reasons the doc explains for eschewing deletions as a feature still hold true in my opinion. You rightly point out that we are investing in extensions mechanisms (KRM functions), and these can be used to create behaviour that is out of scope for Kustomize core, which could include deletion behaviour or even something helm-lifecycle-specific.

what Kustomize plays a role in "life-cycle" operations as shown in this example

Kustomize is strictly focussed on client-side configuration management. The tool itself is not involved in or aware of lifecycle operations, and that increase in scope is not an option for us. For the helm example in particular, perhaps there is a feature that could make sense on the helm transformer, once it is extracted out of Kustomize per #4401 .

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.