Kustomizations introduced by components are not applied to adjacent generated resources

[djmcgreal-cc]

What happened?

See reproduction where generated resource is not modified by an adjacent component whereas the non-generated adjacent resource is.

What did you expect to happen?

That the generated resource would behave the same as the non-generated resources.

How can we reproduce it (as minimally and precisely as possible)?

kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - resource.yaml

components:
  - component

configMapGenerator:
  - name: generated-resource
    literals:
      - key=value
    options:
      disableNameSuffixHash: true

resource.yaml:

apiVersion: v1
kind: ConfigMap
metadata:
  name: adjacent-resource
data:
  blah: hello

component/kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1alpha1
kind: Component
nameSuffix: -modified

Expected output

apiVersion: v1
data:
  blah: hello
kind: ConfigMap
metadata:
  name: adjacent-resource-modified
---
apiVersion: v1
data:
  key: value
kind: ConfigMap
metadata:
  name: generated-resource-modified

Actual output

apiVersion: v1
data:
  blah: hello
kind: ConfigMap
metadata:
  name: adjacent-resource-modified
---
apiVersion: v1
data:
  key: value
kind: ConfigMap
metadata:
  name: generated-resource

Kustomize version

v5.0.1, also {Version:kustomize/v4.5.5 GitCommit:daa3e5e2c2d3a4b8c94021a7384bfb06734bcd26 BuildDate:2022-05-20T20:21:22Z GoOs:darwin GoArch:amd64}

Operating system

MacOS

[natasha41575]

/triage accepted

I think that components should be applied to both resources declared under resources and those that are generated.

/assign @koba1t

[djmcgreal-cc]

We also discovered that generators apply their name translations to helm manifests whereas when specified inline as [configMap|secret]Generators, manifests referencing the generated resources that are loaded through helm are not modified. Though we then found that we cannot resolve the circumstance described in the initial post by using generators, I wonder if it speaks to a wider issue with loading ordering?

[djmcgreal-cc]

Oh, there's already a PR so maybe not :)

[annasong20]

I found the question that this issue spawns of "Why are components run after resources but before all generators and transformers?" very interesting. I ended up reading the components KEP and figured I'd share and collect my thoughts here.

Kustomize intentionally processes components immediately after resources. I assume that this is because the authors proposed components to specify different operations on the same set of resources, and so it was natural to process components after the target resources. The authors probably also wanted to replicate the kustomization experience for components by allowing users to add additional generators and apply transformers to the components.

As pointed out by this issue, there are difficulties with the current processing order. I imagine potential complaints with processing components after generators and transformers, as in #5170, might be

• if the output of generators is needed by all components, it should be in resources
• if the output of generators is needed by particular components, it should be specified in those components
• users need to move generators that they don't want transformed by components to an overlay, thus splitting their application across multiple kustomize layers
• users need to move transformers they want to apply on top of components to an overlay, thus splitting their application across multiple layers

I don't know which processing order will make the most sense, but I guess we can experiment moving around components, given that the feature is still in alpha.

[ephesused]

Fundamentally, I think the question on sequence relates to one's vision on what constitutes the base configuration. My inference is that the original designers did not see generated and transformed resources as part of the base, so that's why components were processed prior to generators and transformers. If generated and transformed resources are considered part of the base, then moving components after generators and transformers makes sense.

I think that any sequence choice is going to have confusing aspects. The key is to ensure the choice is documented and explained, so that users can understand what to expect.

This change is a breaking one for our kustomizations. For example, we have configmaps defined in a component that are merged in a generator in the top-level kustomization.yaml. We will have to invert that pattern (or adjust in some other way) in order to move to a newer version of kustomize.

[natasha41575]

Thank you @ephesused for sharing your perspective, it is very helpful to hear how users are using components. I agree that the sequence will be confusing in any case, and that we need to document it better.

I see generators as part of the base. With #4402 in particular, we will be required to see resources and generators as the same, as resources itself will become a generator, which is needed for the upcoming Composition API. That issue will also make components a generator, so it is not so much about the order of resources, components, and generators per se (as they will all be generators), but we will still need to decide on a reasonable default ordering. We discussed this and feel that it makes sense to see resources from resources the same as resources from other generators, so have elected to move components to running after generators for now - so the order will be resources, then all other generators, then components. Transformers will be applied last, on top of all of these generated resources. We made this change very cautiously and with a lot of discussion and thought, as we want to avoid causing friction for users migrating. But given that Components is in alpha, I think now is the time to make such breaking changes if we are ever going to do it.

That said, we do have someone working on #4402, which will make the ordering of resources, components, and other generators customizable. My hope is that we can have that available soon, to unblock anyone who needs this feature to update.