globbing/wildcards in "resources" field

[ahmetb]

It looks like if I have tens of YAML files in a directory, my kustomization.yaml will exhaustively list them like this:

resources:
- secret.yaml
- service.yaml
- deployment.yaml

It feels like a bit inconvenience. Is it possible to use globs (i.e. **.yaml) for this field? (Skaffold does support this and it's helpful.)

I think if I use kustomize and add new YAML files, aside from kubectl apply -R -f .dir working out of the box, I now have to go back and edit kustomization.yaml for it to work (in other words: two places to update).

[tkellen]

As a recent first time user I was reaching for this immediately as well.

[mxey]

This is a duplicate of #107

[ahmetb]

I can closr in favor of #107 although this has a bit more rationale/explanation.

[Liujingfang1]

Let's keep this one.

[Liujingfang1]

kustomize uses a Loader interface to load the resource content. The loader interface can support both file system loader and remote link loader(#32)
Current interface doesn't support wildcards. We can add it for only file system loader.
Here is a possible approach, in Load function at https://github.com/kubernetes-sigs/kustomize/blob/master/pkg/loader/fileloader.go#L75,
determine if the path has wildcards. If it has, read all the matching files and append the content to the same byte slice with --- separator.

[Liujingfang1]

This is fixed. You can put all the resource files into a resource folder. Then in kustomization.yaml, use

resources:
- resource/*.yaml

This also works for patches

[Place1]

this was randomly removed in #217 what's up with that.

[Place1]

I just found the docs on why it was removed. It does feel overly forceful to push this onto users who may/may-not agree with what this project believes to be a best practice.

This is a real hit to users in my opinion because it complicates the tool. Now users who are used to adding globs to config files need to learn kustomize's specific tooling for editing a file.

If a user really wanted to follow the goals mentioned in the docs then they could simply use files=(./k8s/*) echo "${files[@]}" to get the list themselves...

[Liujingfang1]

@Place1 We recommend users to use kustomize edit add resource command to add files from a directory, globbing is supported in this command. Then all the files matching a glob will be added to kustomization.yaml file.

[aiman-alsari]

@Liujingfang1 I don't think that's a great solution, if we have a hierarchy of kustomization.yamls that would mean we have to traverse up each kustomization.yamls "base" and kustomize edit add resource on those as well.

[tammersaleh]

I agree with @aiman-alsari - I don't think this matches real-world usage. Any chance we could reconsider the removal of resource globbing support?

[jshearer]

This seems like a fundamental flaw. While I understand the spirit of the decision, the reality is that it unnecessarily complicates writing kustomize files becuase not only do I have to learn a new file syntax, I also have to learn a new CLI just to include new files/folders?

The concept of tool fatigue is real -- the k8s operation space is already crowded as it is, don't make people learn extra tools unless there's a really good reason, and this does not seem like one...

[tkellen]

This seems like a fundamental flaw. While I understand the spirit of the decision, the reality is that it unnecessarily complicates writing kustomize files becuase not only do I have to learn a new file syntax, I also have to learn a new CLI just to include new files/folders?

This is not a fundamental flaw. It's a design choice consistent with the ideology of the tool. The reasoning is thoroughly documented in the repo.

I wanted this when I started using kustomize too. But, after a year of daily use I've come to the conclusion this isn't a real problem. If one can't stand the practically effortless task of typing a few filenames, chances are strong one is the kind of person who can easily find a way around it (e.g. find or write a command that does it).

Finally, there is real value in having no chance of unintended side effects (e.g. someone dropping a file in a folder without knowing it will impact a kustomization file somewhere that has a globbing pattern in it).

[jshearer]

Finally, there is real value in having no chance of unintended side effects (e.g. someone dropping a file in a folder without knowing it will impact a kustomization file somewhere that has a globbing pattern in it).

I guess I don't really see this as an issue, since I only ever run kustomize build in CI as part of an automated build process based on a git repo, so I already have to explicitly decide that I want every file in the repo, and while I can understand that some users might not use this repeatable workflow, it seems kind of sad that people in my situation should have to do extra work to work around it.

I guess it just feels smelly similar to a database with duplicate data in it feels smelly: if I have files in folders committed to git, it feels clunky to manually push a button to make kustomize pick up these files.

[vanufryiuk]

This is not a fundamental flaw. It's a design choice consistent with the ideology of the tool. The reasoning is thoroughly documented in the repo.

I wanted this when I started using kustomize too. But, after a year of daily use I've come to the conclusion this isn't a real problem. If one can't stand the practically effortless task of typing a few filenames, chances are strong one is the kind of person who can easily find a way around it (e.g. find or write a command that does it).

Finally, there is real value in having no chance of unintended side effects (e.g. someone dropping a file in a folder without knowing it will impact a kustomization file somewhere that has a globbing pattern in it).

In practice, lack of globs support doesn't help to avoid unintended side effects. It forbids having intended side effects, and more - creates one more place to mistake. I really don't want do add each of my 64 system level manifests to kustomization.yaml. Also, I really do want to deploy all of my 20+ microservices automatically, just by adding/removing one more build artifact to the release definition, instead of cloning the repo and typing 4 more manifest paths in the kustomization.yaml.

[bzon]

This is so sad. I ended up building a refresh script...

#!/bin/bash -e
# refresh.sh
# Create a kustomization.yaml and then add all kubernetes YAMLs into resources...
echo -n "
# built by ./refresh.sh
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
" > kustomization.yaml

yamls=$(find . -type f -name "*.yaml" | grep -v kustomization.yaml)

for y in ${yamls[@]}; do
  kustomize edit add resource $y
done

[aodj]

To add my use case, I ended up here, and don't have kustomize installed since it's invocation is now part of kubectl.

I'm already working around Helm/Tiller to render out Helm charts locally, so having yet another required tool simply to handle globbing is frustrating.

[tprobinson]

Would it be possible to get this feature re-added as an extra key? It would satisfy everyone to keep the current behavior of bases, patches, and resources and add additional keys like baseGlobs, resourceGlobs, and patchGlobs that do allow wildcards.

To be clear, I don't like this solution, if it were me I'd rather just allow globs in the keys themselves and I personally agree with everything that's been said in this thread so far that's in support of reinstating globs.

But I recognize that people and organizations have their reasons for changing things, and generally don't go back on that for a variety of reasons. A compromise like this would allow users who really want the glob feature to "opt in" to using it, while keeping the arguments of avoiding unintended side effects for the users who don't.

As an alternate compromise, there could be a top-layer key like allowGlobs: true, or a CLI flag like --allow-globs.

[gregorybrzeski]

@tprobinson I support your idea of higher layer switch/flag which would enable globing, it would satisfy all parties in this discussion without being a breaking change. The naming of this flag is important as well as it should allow for control of globing for other keys in future. More granular control than allow/deny on all chosen keys.

I believe adding additional keys like baseGlobs adds additional clutter which should be avoided.

[i78]

+1 for simply re-introducing globbing/wildcards or providing a switch to turn it back on.

[ghost]

+1 for me as well

[i78]

I would be happy to submit a PR for that if one of the maintainers would be so kind to express acceptance towards that approach.

[so-jelly]

i was unaware of the kustomize edit subcommand, nice:

for an operator with multiple apis
kustomize edit add resource crds/*crd.yaml

no helper script needed @bzon

[NixBiks]

I was hoping to be able to do something like this

resources:
- ../base/**/development

instead of

resources:
- ../base/some-app/development

There is no way to do that, right?

[towens]

On an overlay:
thing/overlays/dev/kustomization.yaml

That looks like:

---
bases:
  - ../../base
namespace: argocd
resources:
  - applications/httpbin.yaml
configMapGenerator:
  - name: argocd-cm
    behavior: merge
    literals:
      - url=https://argocd.thing.com

When running:
kustomize edit add resource applications/*.yaml

Turns into:

namespace: argocd
resources:
- applications/httpbin.yaml
- ../../base
configMapGenerator:
- behavior: merge
  literals:
  - url=https://argocd.thing.com
  name: argocd-cm
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

The bases key is removed and the value is merged with resources. Something I am missing?

[Shell32-Natsu]

@towens bases is replaced by resources in newer version of kustomize.

[towens]

@Shell32-Natsu understood, thanks. The local kustomize version is ahead of what kubectl has. kubectl -k is pretty nice. Redirecting kustomize build isn't a deal breaker.

[simplenotezy]

This is fixed. You can put all the resource files into a resource folder. Then in kustomization.yaml, use

resources:
- resource/*.yaml

This also works for patches

When I am using this, in a folder called "k8s" I get error:

Error: accumulating resources: accumulating resources from 'k8s/*.yaml': evalsymlink failure on '/home/runner/work/project-k8s/project-k8s/k8s/*.yaml' : lstat /home/runner/work/project-k8s/project-k8s/k8s/*.yaml: no such file or directory
error: no objects passed to apply
Error: Process completed with exit code 1.

Config:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- k8s/*.yaml

[seh]

Note this comment: #119 (comment). This globbing feature was there briefly, but now it's gone again.

[arash-bizcover]

#3205

[xdoncalvox]

Was this feature ever added back?
Will be?

[migueleliasweb]

Still not present in 2023.

$ kustomize version
v5.0.0

Can we bring this back? It's a nightmare when you have multiple overlays and you're adding more patches or resources...

Pretty please 🙏

[thiDucTran]

This is so sad. I ended up building a refresh script...

#!/bin/bash -e
# refresh.sh
# Create a kustomization.yaml and then add all kubernetes YAMLs into resources...
echo -n "
# built by ./refresh.sh
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
" > kustomization.yaml

yamls=$(find . -type f -name "*.yaml" | grep -v kustomization.yaml)

for y in ${yamls[@]}; do
  kustomize edit add resource $y
done

thanks for this. we changed it a bit since we have existing patches in our base's kustomization.yaml file

#!/bin/bash
# Inspired by author of https://github.com/kubernetes-sigs/kustomize/issues/119#issuecomment-526234246
echo -n "
resources:
" >> kustomization.yaml

yamls=$(find . -type f -name "*.yaml" | sort -bf | grep -v kustomization.yaml)

for yaml_file in ${yamls[@]}; do
  kustomize edit add resource $yaml_file
done

our base's kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
secretGenerator:
- name: grafana-datasources
  files:
    - datasources.yaml=grafana-datasources/grafana-dashboardDatasources.txt
  type: "Opaque"
  namespace: monitoring
  behavior: replace
configMapGenerator:
# https://grafana.com/grafana/dashboards/15757-kubernetes-views-global/
- name: grafana-dashboard-kuberenetes-views-global
  files:
    - kuberenetes-views-global.json=grafana-dashboards/kuberenetes-views-global.json
  namespace: monitoring
# https://grafana.com/grafana/dashboards/9614-nginx-ingress-controller/
- name: grafana-dashboard-nginx-ingress-controller
  files:
    - nginx-ingress-controller.json=grafana-dashboards/nginx-ingress-controller.json
  namespace: monitoring  
patches:
  - target:
      version: v1
      group: apps
      kind: Deployment
      namespace: monitoring
    patch: |-
      kind: not-important
      metadata:
        name: not-important    
      spec:
        template:
          spec:
            affinity:
              nodeAffinity:
                requiredDuringSchedulingIgnoredDuringExecution:
                  nodeSelectorTerms:
                  - matchExpressions:
                    - key: kubernetes.azure.com/mode
                      operator: In
                      values:
                      - user
  - target:
      version: v1
      group: apps
      kind: Deployment
      namespace: monitoring
      name: grafana
    patch: |-
      - op: add
        path: /spec/template/spec/containers/0/env/-
        value:
          name: GF_PATHS_CONFIG
          value: /etc/grafana/grafana.ini
      - op: add
        path: /spec/template/spec/containers/0/volumeMounts/-
        value:
          mountPath: /etc/grafana/certs
          name: omaticcloud-io-wildcard-tls
          readOnly: false
      - op: add
        path: /spec/template/spec/volumes/-
        value:
          name: omaticcloud-io-wildcard-tls
          secret:
            secretName: omaticcloud-io-wildcard-tls-grafana
      - op: add
        path: /spec/template/spec/containers/0/volumeMounts/-
        value:
          mountPath: /grafana-dashboard-definitions/0/kuberenetesviewsglobal
          name: grafana-dashboard-kuberenetes-views-global
          readOnly: false
      - op: add
        path: /spec/template/spec/volumes/-
        value:
          name: grafana-dashboard-kuberenetes-views-global
          configMap:
            name: grafana-dashboard-kuberenetes-views-global
      - op: add
        path: /spec/template/spec/containers/0/volumeMounts/-
        value:
          mountPath: /grafana-dashboard-definitions/0/nginxingresscontroller
          name: grafana-dashboard-nginx-ingress-controller
          readOnly: false
      - op: add
        path: /spec/template/spec/volumes/-
        value:
          name: grafana-dashboard-nginx-ingress-controller
          configMap:
            name: grafana-dashboard-nginx-ingress-controller             
  - target:
      version: v1
      group: rbac.authorization.k8s.io
      kind: ClusterRole
      name: prometheus-k8s
    patch: |-
      apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRole
      metadata:
        name: prometheus-k8s
      rules:
      - apiGroups:
        - '*'
        resources:
        - '*'
        verbs:
        - '*'
      - nonResourceURLs:
        - '*'
        verbs:
        - '*'

[all2ham]

+1 for globbing

[natasha41575]

The decision was made before my time but globbing is specifically and intentionally eschewed: https://kubectl.docs.kubernetes.io/faq/kustomize/eschewedfeatures/#globs-in-kustomization-files

And as a result, I don't think we can support it. The recommendation is to run kustomize edit add resource * before kustomize build. I know this will upset users and I do genuinely empathize with those who are affected by this inconvenience, but kustomize has a very specific scope and philosophy that it was designed around and the current maintainers would like to stick to that original intention.

An older maintainer of kustomize left a lovely comment explaining why we stick so closely to these principles which I will repeat here:

Our guiding principles aren't just dogma for its own sake–they're promises we make to our users about how Kustomize works. They define what distinguishes it among the vast array of options in the configuration management landscape, and they help us keep making a great tool with a consistent vision and user experience across time and leadership changes. We regularly get feature requests to reverse every single one of these principles, and I'm sure if we accommodated them, many users would find those changes useful as well. However, it would likely break workflows for those taking advantage of the distinguishing features Kustomize was designed to provide, and it would certainly degrade Kustomize's user experience in the long run.

That said, I think we should be able to make it possible for kustomize edit to support most use cases that require globbing. I think we have one request for kustomize edit to accept a path to a kustomization.yaml file or even have a recursive option, both of which I think are reasonable and we can consider more deeply. We are happy to understand what use cases our users have, but we need to design solutions that are in line with kustomize's goals, which unfortunately globbing is not.

[rileymcdowell]

I don't know why this was locked and then unlocked, but I'll take this opportunity to provide my two cents on the issue. @natasha41575 pointed out the documentation of the decision. I would like to respond. I'll copy the relevant parts here:

kustomize supports the best practice of storing one’s entire configuration in a version control system.

Globbing the local file system for files not explicitly declared in the kustomization file at kustomize build time would violate that goal.

I think I understand the concern expressed in this documented decision, but I think it's a little misguided. The k8s yaml files I want to glob are in a version control system right alongside my kustomization.yaml file, in the same directory even. And since, by kustomize's default, all the files must be deeper in the directory hierarchy than the location of my kustomization.yaml file, they're guaranteed to exist somewhere in my repository rather than outside it in an uncontrolled file system like the rationale implies.

Allowing globbing in a kustomization file would also introduce the same problems as allowing globbing in ... BUILD/Makefile dependency rules.

It's quite useful, dare I say ubiquitous, to use globbing or other wildcard expansion techniques in Makefiles to create a dependency between all source code files and their corresponding object files based on file extension naming conventions, and likewise for an executable that depends on compile object files. The documentation of this decision implies that using wildcards is a bad practice or creates problems, but I don't see where there's any evidence of that. In fact it seems to be quite effective in the Makefile example.

I would humbly request that you revisit the rationale for the specifically and intentionally eschewed globbing feature and reflect on whether or not the rationale for its exclusion is still relevant.

[seh]

The documentation of this decision implies that using wildcards is a bad practice or creates problems, but I don't see where there's any evidence of that. In fact it seems to be quite effective in the Makefile example.

Alongside Makefiles, the documentation mentions Bazel's BUILD.bazel files. While Bazel does allow globbing via its glob function, using it isn't the norm, and the documentation deems only a subset of their possible use "acceptable". Instead, Bazel encourages BUILD.bazel file authors to list their input files explicitly. There are some language-specific tools that will discover these input files and edit BUILD.bazel files accordingly.

[AmiditeX]

Our guiding principles aren't just dogma for its own sake–they're promises we make to our users about how Kustomize works. They define what distinguishes it among the vast array of options in the configuration management landscape, and they help us keep making a great tool with a consistent vision and user experience across time and leadership changes. We regularly get feature requests to reverse every single one of these principles, and I'm sure if we accommodated them, many users would find those changes useful as well. However, it would likely break workflows for those taking advantage of the distinguishing features Kustomize was designed to provide, and it would certainly degrade Kustomize's user experience in the long run.

I must disagree on that. I've been following that issue for years (!!) now. We are not using Kustomize in production because it is fundamentally incompatible with our GitOps workflows. Declaring a new ressource in the kustomization.yaml for EVERY new file we add to our Git is impractical at our scale. Which is funny considering the link you've quoted about eschewed features says : kustomize supports the best practice of storing one’s entire configuration in a version control system. I do support that best practice too, and kustomize is actively FIGHTING against me when I try to integrate it to our ArgoCD. We have hundred of base resources for a single app and kustomize won't allow us to add anything automatically with globing.

The files are there though, defined in the Git, readable by everyone. And the globing in the kustomization.yaml would indicate that any file in that folder SHOULD be applied. That seems to fit the "best practices" kustomize is pushing.

I honestly don't see how it is a best practice to physically reference every single file I want in my base or resources. This feels like dogma. How would it break anyone's workflow to integrate globbing?

[phoenix-bjoern]

To create a fresh kustomization.yaml for a directory (e.g. created by helm template --output-dir) kustomize create --autodetect --recursive does the trick for us.

[Elyytscha]

Our guiding principles aren't just dogma for its own sake–they're promises we make to our users about how Kustomize works. They define what distinguishes it among the vast array of options in the configuration management landscape, and they help us keep making a great tool with a consistent vision and user experience across time and leadership changes. We regularly get feature requests to reverse every single one of these principles, and I'm sure if we accommodated them, many users would find those changes useful as well. However, it would likely break workflows for those taking advantage of the distinguishing features Kustomize was designed to provide, and it would certainly degrade Kustomize's user experience in the long run.

I must disagree on that. I've been following that issue for years (!!) now. We are not using Kustomize in production because it is fundamentally incompatible with our GitOps workflows. Declaring a new ressource in the kustomization.yaml for EVERY new file we add to our Git is impractical at our scale. Which is funny considering the link you've quoted about eschewed features says : kustomize supports the best practice of storing one’s entire configuration in a version control system. I do support that best practice too, and kustomize is actively FIGHTING against me when I try to integrate it to our ArgoCD. We have hundred of base resources for a single app and kustomize won't allow us to add anything automatically with globing.

The files are there though, defined in the Git, readable by everyone. And the globing in the kustomization.yaml would indicate that any file in that folder SHOULD be applied. That seems to fit the "best practices" kustomize is pushing.

I honestly don't see how it is a best practice to physically reference every single file I want in my base or resources. This feels like dogma. How would it break anyone's workflow to integrate globbing?

at scale we automate gitops with cicd, which means we create cicd which commits automatically in git repos handled by argocd

thats why i'm fine with using

kustomize edit add resource (prefix)*(sufix).y*ml

[arusa]

I don't understand the reason behind removing this option. The present and future of Kustomize is being used in CI/CD pipelines that run ONLY the code that they pull from git. So there is no danger of someone putting random files into the local filesystem.

When trying to automate everything it's easy to add and delete .yaml files to a git repo (for example a simple API request to GitLab). But it's hard to clone the whole repo, add a file, then update the kustomization.yaml and commit everything and when you want to delete this file again, you also have to find the corresponding line in the kustomization.yaml and remove it. Completely unnecessary complexity.

The whole idea of linters is to have a way to force individual best practice guidelines instead of completely removing possibilities and breaking existing workflows.

[thecodeassassin]

I don't understand the reason behind removing this option. The present and future of Kustomize is being used in CI/CD pipelines that run ONLY the code that they pull from git. So there is no danger of someone putting random files into the local filesystem.

When trying to automate everything it's easy to add and delete .yaml files to a git repo (for example a simple API request to GitLab). But it's hard to clone the whole repo, add a file, then update the kustomization.yaml and commit everything and when you want to delete this file again, you also have to find the corresponding line in the kustomization.yaml and remove it. Completely unnecessary complexity.

The whole idea of linters is to have a way to force individual best practice guidelines instead of completely removing possibilities and breaking existing workflows.

I fully agree, the reason for removing the blog feature wasn't even a good reason. it makes sense from a development perspective but a infrastructure as code perspective. We have some projects with 20-30 yaml files and the kustomization file could be shrunk to 3 lines. It's harder to maintain this way.

This feature needs to be reintroduced, especially because it was removed for the wrong reasons.

[robpearce-flux]

ugh, yes. I've also just ended up here for the same reason. This makes for quite the depressing read. Could we reopen this one please ?

[tprobinson]

Perhaps it's been said before in this thread, but I want to try to elaborate on the real problem here since it is easily misunderstood. Or at least, my perspective on it.

Imagine if you wanted to bake a cake. You need ingredients and a recipe. Conveniently, all the ingredients for a cake are already organized on the second shelf of your pantry.

Being able to put a glob in a Kustomization is like the recipe saying "Use all the ingredients on the second shelf". You looked in one place and found out what you needed to know so you can start baking.

Having to run a command beforehand is like someone saying "look in the cookbook before you read the recipe to see what the ingredients are", and the cookbook just says "Look at the ingredients on the second shelf and use them all". It accomplishes the same thing but with more steps and a whole extra book, for no perceivable gain.

It looks obvious to people that this feature should exist, and at this point it feels simply dogmatic. There's been a lot of interest in this over the years and a lot of attempts to explain why people want this. Kustomization has already had this feature in the past and it was removed, so it's not a matter of implementation being difficult. It doesn't conflict with or alter the existing workflow if people simply don't want to use globs, so it's not a matter of disrupting the existing userbase.

So people are left asking, why doesn't this exist? And no answer has been given other than "have you tried not wanting this feature?" That's why this feels like such a needless debate, and why telling us to use a command feels like such a miscommunication.

[tkellen]

Y'all, this software was written by architectural astronauts pontificating about how to manage complexity. The experiment failed years ago and just happened to be jammed into kubectl before that was realized. Save your breath, time and energy. Use what it has and shell script wrapper hack the rest. This shit is not going to change.