Configuration with custom resource without crd existing yet

[afterlook]

I'm trying to read the code of kustomize-controller and I want to confirm something. Let's say that I deploy to my repository a configuration that consists of some HelmRelease that installs some crd and right next to it I have a custom resource. What is the default behavior then? To my knowledge it will fail, because kustomize reconciliation will error out on kubectl apply ... --dry-run.

Did I miss something? If yes can somebody point it out?
If I didn't miss anything then is this desired behaviour? If so how to deal with such cases?

[afterlook]

bump?

[sloppycoder]

@krystian-kulgawczuk , as you guessed, it fails...

My use case is use kube-prometheus-stack Helm chart to install the Prometheus Operator, which includes bunch of CRDs, including ServiceMontior. Before the Helm chart is installed, any Kustomization that refers to ServiceMonitor will fail validation, example message below:

│ Events:                                                                                                                                                                           │
│   Type    Reason  Age    From                  Message                                                                                                                            │
│   ----    ------  ----   ----                  -------                                                                                                                            │
│   Normal  error   2m37s  kustomize-controller  validation failed: unable to recognize "77c4c3a6-1886-4ca8-94c1-f95dc9f4de07.yaml": no matches for kind "PrometheusRule" in versio │
│ n "monitoring.coreos.com/v1"                                                                                                                                                      │
│ unable to recognize "77c4c3a6-1886-4ca8-94c1-f95dc9f4de07.yaml": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"                                       │
│                                                                                                                                             

This is my directory structure, which does not work.

.
├── kustomization.yaml
├── kube-prometheus-stack
│   ├── kube-prometheus-stack-values.yaml
│   ├── kustomization.yaml
│   ├── kustomizeconfig.yaml
│   ├── namespace.yaml
│   ├── release.yaml             <-- HelmRelase that installs Prometheus Operator CRDs
└── traefik-monitoring
    ├── kustomization.yaml
    └── traefik-servicemonitor.yaml  <-- refers to a CRD installed by above HelmRelease

[yebyen]

When you're pulling a manifest from a trusted upstream like Prometheus, that includes CRDs mixed together with instances of the CRD, this is a good example of a case where you can usually trust they are valid, and simply isolate them together away from the rest of your manifests, in a Flux Kustomization by themselves. This makes it safe enough to disable the validation, of course YMMV but that's what I would recommend, rather than trying to split them apart and manage your own fork of Prometheus manifests. 👍

[yebyen]

If you really want to do what you're describing, you can disable apply-time validation and ensure that validation is done some other way, like CI checks such as validate.sh as shown in the flux2-helm-kustomize examples.

Check out in the Kustomization spec validate which can be set to none

This has the risk that any invalid configuration you promote will be partially applied without raising any alarms. This is closer to the behavior in flux v1 and sounds like what you are looking to be able to do.

The standard behavior of validating is designed to prevent invalid resources from being partially applied to the cluster, and that means you can't use a CRD before it has been applied. This is by design but it can be worked around in places where it is undesirable to split them up.

[yebyen]

Another example of how to handle this is using dependency ordering, as shown in the flux2-multi-tenancy example repo. I just thought I'd mention this as it is considered the canonical best practices solution, for folks who absolutely want everything validated before applying it to the cluster. It was clear this is not what you wanted but it is what we recommend first.

In the example there, Kyverno Policies cannot be applied before Kyverno is installed. It is handled using dependency ordering with dependsOn — I thought this might be something else you want to take a look at in case you haven't seen it.

[sloppycoder]

@yebyen , thanks very much for the pointer to "dependsOn", works beautifully.