Allow setting every array element in replacements

[yogeshgadge]

On the lines of targeting specific array entry prop.[name=nginx].host I tried entry prop.-.host so that all the array elements are targeted. However this did not work.

See my example below
kustomization.yaml

resources:
- configmap.yaml
- virtual-services.yaml
replacements:
- source:
     kind: ConfigMap
     fieldPath: data.MY_HOST
  target:
    kind: VirtualService
    fieldPaths:
     -  spec.http.-.rpute.0.destination.host

Here are the detailed values https://stackoverflow.com/questions/68322864/kustomize-how-to-target-replacement-in-every-array

I am using kustomize 4.2.0 version

[natasha41575]

There is no mechanism to do so, but we can consider adding one.

@monopole @KnVerey any concerns or objections with a mechanism to target all array elements in a replacement fieldPath?

[natasha41575]

In its current state you would have to list out all the fieldPaths you need.

[k8s-ci-robot]

@yogeshgadge: This issue is currently awaiting triage.

SIG CLI takes a lead on issue triage for this repo, but any Kubernetes member can accept issues by applying the triage/accepted label.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[yogeshgadge]

Using [] or .. for representing all elements seems reasonable to me.
IMO * makes more sense than empty or []

spec.http.*.route.0.destination.host

[KnVerey]

I think the feature makes sense. As for the syntax, fieldPath is essentially using a subset of JSONPath syntax, so we should stick with its way of expressing things as we add features. Specifically, our implementation should be consistent with what k/k packages use, i.e. this parser from client-go if my searching is serving me well. In this case, list[*].

[natasha41575]

Being consistent with k/k sounds good to me. In that case, the syntax would be something like spec.http.[*].route.0.destination.host when implemented.

[natasha41575]

Adding a comment to note a caveat in that fieldPath slightly differs from JSONPath syntax.

fieldPath separates on "." for everything - so while a JSON path is something like spec.http[1].route, the equivalent kustomize replacements path would be spec.http.1.route. In order to be analogous to the current fieldPath syntax, the syntax for setting every array element should actually be spec.http.*.route.

@m-Bilal, feel free to assign yourself if you would like

[m-Bilal]

/assign

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[siegenthalerroger]

Has there been any work on this topic?

[KnVerey]

@m-Bilal are you still planning to work on this?

[m-Bilal]

Sorry I have not been able to work on this yet, been busy with office work. I do plan on working on this, but I'll be occupied this week as well. If anyone else wants to take this up, please feel free to do so. In case nobody takes this up, I'll leave a comment here when I do start working on this.

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[siegenthalerroger]

A maintainer should probably remove the lifecycle/rotten and stale from this, seeing as it's still active and relevant

[m-Bilal]

/remove-lifecycle rotten

[natasha41575]

/help

[k8s-ci-robot]

@natasha41575:
This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[koba1t]

/assign

[natasha41575]

This is fixed by #4424. The only thing left to do is documentation.

@koba1t would you be interested in updating the documentation for this feature? The replacement doc lives here: https://github.com/kubernetes-sigs/cli-experimental/tree/master/site/content/en/references/kustomize/kustomization/replacements

[koba1t]

I'm creating PR to add a document. kubernetes-sigs/cli-experimental#240
Do you think it needed to add an example?

[natasha41575]

Documentation LGTM. Will merge the docs PR when we do a release. Thank you!

[tis-rpage]

I think there's a problem with this code.
kustomize version

{Version:kustomize/v4.5.3 GitCommit:de6b9784912a5c1df309e6ae9152b962be4eba47 BuildDate:2022-03-24T20:51:20Z GoOs:linux GoArch:amd64}

kustomize build test/

Error: wrong Node Kind for spec.template.spec.containers expected: MappingNode was SequenceNode: value: {- name: controller
  image: public.ecr.aws/aws-controllers-k8s/rds-controller:v0.0.19
  imagePullPolicy: IfNotPresent
  env:
  - name: AWS_REGION
  - name: AWS_ENDPOINT_URL
    value: ""
  command:
  - ./bin/controller
  args:
  - --aws-region
  - $(AWS_REGION)
  - --aws-endpoint-url
  - $(AWS_ENDPOINT_URL)
  ports:
  - containerPort: 8080
    name: http
  resources:
    limits:
      cpu: 100m
      memory: 128Mi
    requests:
      cpu: 50m
      memory: 64Mi
  securityContext:
    allowPrivilegeEscalation: false
    capabilities:
      drop:
      - ALL
    privileged: false
    runAsNonRoot: true}

File: test/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: test
  name: test
spec:
  replicas: 1
  template:
    spec:
      containers:
        - name: controller
          image: public.ecr.aws/aws-controllers-k8s/rds-controller:v0.0.19
          imagePullPolicy: IfNotPresent
          env:
            - name: AWS_REGION
            - name: AWS_ENDPOINT_URL
              value: ""
          command:
            - ./bin/controller
          args:
            - --aws-region
            - $(AWS_REGION)
            - --aws-endpoint-url
            - $(AWS_ENDPOINT_URL)
          ports:
            - containerPort: 8080
              name: http
          resources:
            limits:
              cpu: 100m
              memory: 128Mi
            requests:
              cpu: 50m
              memory: 64Mi
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop:
                - ALL
            privileged: false
            runAsNonRoot: true
      hostIPC: false
      hostNetwork: false
      hostPID: false
      nodeSelector:
        kubernetes.io/os: linux
      terminationGracePeriodSeconds: 10

File: test/kustomization.yaml

resources:
  - deployment.yaml

configMapGenerator:
  - name: aws
    options:
      disableNameSuffixHash: true
    literals:
      - AWS_REGION=us-east-1

replacements:
  - source:
      kind: ConfigMap
      name: aws
      fieldPath: data.AWS_REGION
    targets:
      - options: { create: true }
        select:
          kind: Deployment
        fieldPaths:
          - spec.template.spec.containers.*.env.[name=AWS_REGION].value
# this works if uncommented
#  - source:
#      kind: ConfigMap
#      name: aws
#      fieldPath: data.AWS_REGION
#    targets:
#      - options: { create: true }
#        select:
#          kind: Deployment
#        fieldPaths:
#          - spec.template.spec.containers.[name=controller].env.[name=AWS_REGION].value

[natasha41575]

@koba1t could you please investigate the above issue?

[koba1t]

I was thinking of no use case to create nodes when wildcard matching (and PathMatcher didn't support create option).
So, I disable wild card matching when the Create option is enabled, and show the error message if using a wild card and the Create option was enabled.

I feel this bug is caused to output error messages. I can fix it.
So, do you think you need create option when using wild card?

[tis-rpage]

I would very much expect a wildcard and create options to be a supported combination. If you need to add an environment variable or an argument to every pod/deployment/statefulset, then that feature would be required. Without that option, we're only able to modify environment/arguments that are already defined with a new value.

[natasha41575]

I see there is a new issue filed regarding this, so let's move the discussion there.

#4561

[natasha41575]

I left a commend #4561 (comment)

I think we should start by improving the error message, but in general I agree that the feature makes sense to support.