Variable substitution does not work for ConfigMapKeySelector at global Workflow or Step level

[htouloupas]

Summary

Environment

• Latest argo-workflows from master installed from https://raw.githubusercontent.com/argoproj/argo-workflows/master/manifests/quick-start-postgres.yaml
• Kubernetes version: v1.22.2
• Minikube version: v1.23.2
• Mac OS X 11.6

Given the instructions defined here https://github.com/argoproj/argo-workflows/blob/master/docs/variables.md I expect that variables work in every field of a Workflow resource. I am using a ConfigMapKeySelector which has a key that references a global or local step variable. The expansion does not work and the following template is parsed as a string:
{{ workflow.parameters.var }}

Diagnostics

Example ConfigMap applied on the argo namespace:

apiVersion: v1
kind: ConfigMap
metadata:
  name: workflow-template-bug
  namespace: argo
  labels:
    # Note that this label is required for the informer to detect this ConfigMap.
    workflows.argoproj.io/configmap-type: Parameter
data:
  testKey: 'testValue'

Applied Workflows:

metadata:
  name: delightful-python
  namespace: argo
spec:
  arguments:
    parameters:
      - name: reference
        value: testKey
      - name: message
        valueFrom:
          configMapKeyRef:
            name: workflow-template-bug
            key: '{{ workflow.parameters.reference }}'
  entrypoint: argosay
  templates:
    - name: argosay
      inputs:
        parameters:
          - name: message
            value: '{{workflow.parameters.message}}'
      container:
        name: main
        image: 'argoproj/argosay:v2'
        command:
          - /argosay
        args:
          - echo
          - '{{inputs.parameters.message}}'

# or 

metadata:
  name: delightful-python
  namespace: argo
  labels:
    example: 'true'
spec:
  arguments:
    parameters:
      - name: reference
        value: testKey
  entrypoint: argosay
  templates:
    - name: argosay
      inputs:
        parameters:
          - name: message
            valueFrom:
              configMapKeyRef:
                name: workflow-template-bug
                key: '{{ workflow.parameters.reference }}'
      container:
        name: main
        image: 'argoproj/argosay:v2'
        command:
          - /argosay
        args:
          - echo
          - '{{inputs.parameters.message}}'

The error messages I get when submitting this Workflow from the UI are:

Failed: failed to set global parameters: failed to set global parameter message from configmap with name workflow-template-bug and key {{workflow.parameters.reference}}: ConfigMap 'workflow-template-bug' does not have the key '{{workflow.parameters.reference}}'

and for the second Workflow:

unable to retrieve inputs.parameters.message from ConfigMap: ConfigMap 'workflow-template-bug' does not have the key '{{ workflow.parameters.reference }}'

Is this a problem with nested fields or doesn't the ConfigMapKeySelector work with dynamic variable substitution?

Linking to #6869

---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.

[terrytangyuan]

This might have been fixed by @smile-luobin in #6898 but need to double check.

[sarabala1979]

Failed: failed to set global parameters: failed to set global parameter message from configmap with name workflow-template-bug and key {{workflow.parameters.reference}}: ConfigMap 'workflow-template-bug' does not have the key '{{workflow.parameters.reference}}'

@terrytangyuan @smile-luobin
The above error is coming from CLI/Server validation, {{}} parameter substitution will happen during the workflow process. CLI/Server validation should consider this scenario as a placeholder and pass the validation.

[sarabala1979]

@terrytangyuan I will take back my comments. It is happening in operate function. I kind of think about how we can fix this. it is a kind of chicken egg problem. setGlobalParameters and substituteGlobalVariables need to go sequentially with one after another one. We can document it Configmap key as a parameter substitution can't be supported.

[sarabala1979]

@terrytangyuan One alternative, we can have substitute reference in setGlobalParameters if variable refer to another variable. we can check the map and assign it. what is your thought?

[yriveiro]

I'm hitting this limitation also, and in my case, my template uses the configMapKeyRef ref to a config map that is different between runs.

If I can not make it dynamically I don't know how to tell the template to load a config map depending on an argument parameter.

Is there any workaround to this, or a correct way to do it in Argo?.

Example of what I'm doing:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: workflow-template-simulation-
spec:
  entrypoint: dag-simulation
  serviceAccountName: argo
  arguments:
    parameters:
    - name: config
      value: false
  templates:
    - name: watchdog-config
      inputs:
        parameters:
          - name: container
            valueFrom:
              configMapKeyRef:
                name: '{{workflow.parameters.config}}'
                key: container
      script:
        image: busybox
        command: [sh]
        source: |
         ....

[terrytangyuan]

@terrytangyuan One alternative, we can have substitute reference in setGlobalParameters if variable refer to another variable. we can check the map and assign it. what is your thought?

I am okay with this. Checking configmaps is not expensive since we have our own informer now.

[sarabala1979]

@terrytangyuan I mean woc.globalParams if the value has {{ we can check the woc.globalParams and get the value.

[smile-luobin]

@terrytangyuan I will take back my comments. It is happening in operate function. I kind of think about how we can fix this. it is a kind of chicken egg problem. setGlobalParameters and substituteGlobalVariables need to go sequentially with one after another one. We can document it Configmap key as a parameter substitution can't be supported.

Maybe this is the limitation of parameter substitution. As far as i know, parameter substitutions can be done succeeded in these common scenarios:

• the value of inputs parameters in template;
• the command of container in template;
• the args of container in template;
• other fields in template;

But other scenarios are not supported:

• key and name of configMapKeySelector;
• storage of volumeClaimTemplates;
• other fields (except template) in workflow;

I think it better to document these supported scenarios of parameter substitution.

[sarabala1979]

@smile-luobin Do you like to submit the documentation PR for this?

[terrytangyuan]

As @sarabala1979 mentioned, the approach would be to check globalParam map and replace any {{ placeholders #7061 (comment).

Would anyone like to submit a PR for this?