inputs or workflow is nil when used in sprig expression that has more than 1 parameter (withParam loop)

[aaaaahaaaaa]

Checklist

• Double-checked my configuration.
• Tested using the latest version.
• Used the Emissary executor.

Summary

What happened/what you expected to happen?

With the following context:

• within a withParam loop
• in argument.parameter.value
• using a sprig function that as more than 1 parameter (e.g. sprig.replace)
• referencing workflow.parameters.paramX or inputs.parameters.paramX

Then, the following error is thrown:

  Warning  WorkflowFailed   3s    workflow-controller  failed to evaluate expression: cannot fetch parameters from <nil> (1:23)
 |  sprig.replace(inputs.parameters.paramX, 'YYY', 'Replace XXX') 
 | ......................^

If the expression uses a sprig function with 1 parameter, then the error doesn't occur.

What version are you running?

3.3.0

Diagnostics

Paste the smallest workflow that reproduces the bug. We must be able to run the workflow.

---
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: my-template
  namespace: argo
spec:
  entrypoint: execute

  arguments:
    parameters:
      - name: paramX
        value: XXX
      - name: paramY
        valueFrom:
          configMapKeyRef:
            name: my-config
            key: my-key

  templates:
    - name: execute
      steps:
        - - name: other-template
            templateRef:
              name: other-template
              template: execute
            withParam: "{{workflow.parameters.paramY}}"
            arguments:
              parameters:
                - name: my-param
                  value: "{{= sprig.replace(workflow.parameters.paramX, "YYY", "Replace XXX" }}"

All the following are however working and are producing the expected values:

              parameters:
                - name: my-param
                  value: "{{ workflow.parameters.paramX }}"

              parameters:
                - name: my-param
                  value: "{{= workflow.parameters.paramX }}"

              parameters:
                - name: my-param
                  value: "{{= sprig.trim(workflow.parameters.paramX) }}"

---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.

[aaaaahaaaaa]

Could be related to #6790? @alexec

[sarabala1979]

@aaaaahaaaaa Is it only happening when you are using input.parameters.xxxx?

[aaaaahaaaaa]

@sarabala1979 Or workflow.parameters as mentioned above.

[aaaaahaaaaa]

@sarabala1979 I did some more testing and I think I drilled down to a more specific diagnostic that my example above doesn't show.

The problem seems to occur when both worflow.parameters.XXX and item.YYY are used next to each other as parameters of the same function (as part of a withParam loop).

Examples:

WORKS

{{= sprig.replace('what ever', workflow.parameters.XXX, workflow.parameters.XXX) }}

WORKS

{{= sprig.replace('what ever', item.YYY, item.YYY) }}

ERROR cannot fetch parameters from <nil>...

{{= sprig.replace('what ever', workflow.parameters.XXX, item.YYY) }}

This seems really strange but I believe there's really a bug here.

[tacf]

Hi, bumped into an issue that i think it may relate to this.

There's a weird behaviour when using both workflow. and inputs. in an expression. In my case i'm trying to dynamically set a mutex, below i'll leave the examples and the calculated mutex value to see if it helps trying to pin point this.

workflow.parameters.XXX = true
inputs.parameters.YYY = false

"{{=workflow.parameters.XXX}}" # argo/Mutex/true
"{{=workflow.parameters.XXX == 'true' ? workflow.uid : workflow.parameters.XXX }}" # argo/Mutex/d003efcc-7817-4dc0-b435-5c98640ac358
"{{=inputs.parameters.YYY == 'false' ? workflow.uid : 'falseee' }}" # argo/Mutex/{{=inputs.parameters.YYY == 'false' ? workflow.uid : 'falseee' }}
"{{inputs.parameters.YYY}}" # argo/Mutex/false
"{{=inputs.parameters.YYY}}" # argo/Mutex/false
"{{=workflow.parameters.XXX == 'true' and inputs.parameters.YYY == 'false' ? 'true' : 'false'}}" # argo/Mutex/{{=workflow.parameters.XXX == 'true' and inputs.parameters.YYY == 'false' ? 'true' : 'false'}}
"{{=inputs.parameters.YYY == 'false' and workflow.parameters.XXX == 'true' ? 'true' : 'false'}}" # argo/Mutex/{{=inputs.parameters.YYY == 'false' and workflow.parameters.XXX == 'true' ? 'true' : 'false'}}
"{{=inputs.parameters.YYY == 'false' and inputs.parameters.YYY == 'true' ? 'true' : 'false'}}" # argo/Mutex/false
"{{=workflow.parameters.XXX == 'true' and workflow.parameters.XXX == 'false' ? 'true' : 'false'}}" # argo/Mutex/false 

As you can see in the case that both workflow. and inputs. is used the mutex value goes as the literal string of the expression, and according to the tests i did is not just a visual thing (it could be either an UI / Logs issue no rendering the calculated value) the workflow is actually locking on the expression string.

Let me know if you find this is related or should i move this to a new issue.

[alexec]

I don't think you can use "and", you should use "&&".

[tacf]

Unless this is a custom configuration on argo side Expr does support this

Edit: This can easily be tested by something like {{= 'X' == 'X' and 'Y' == 'Y' ? ... }} which works just fine.

[alexec]

The example workflow is invalid.

[alexec]

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: my-wf-
spec:
  entrypoint: main

  arguments:
    parameters:
      - name: XXX
        value: "true"
      - name: YYY
        value: "false"

  templates:
    - name: main
      inputs:
        parameters:
          - name: XXX
            value: "{{workflow.parameters.XXX}}"
          - name: YYY
            value: "{{workflow.parameters.YYY}}"
      container:
        image: argoproj/argosay:v2
        command:
          - sh
          - -c
        args:
          - |
            echo "{{=workflow.parameters.XXX}}" # argo/Mutex/true
            echo "{{=workflow.parameters.XXX == 'true' ? workflow.uid : workflow.parameters.XXX }}" # argo/Mutex/d003efcc-7817-4dc0-b435-5c98640ac358
            echo "{{=inputs.parameters.YYY == 'false' ? workflow.uid : 'falseee' }}" # argo/Mutex/{{=inputs.parameters.YYY == 'false' ? workflow.uid : 'falseee' }}
            echo "{{inputs.parameters.YYY}}" # argo/Mutex/false
            echo "{{=inputs.parameters.YYY}}" # argo/Mutex/false
            echo "{{=workflow.parameters.XXX == 'true' and inputs.parameters.YYY == 'false' ? 'true' : 'false'}}" # argo/Mutex/{{=workflow.parameters.XXX == 'true' and inputs.parameters.YYY == 'false' ? 'true' : 'false'}}
            echo "{{=inputs.parameters.YYY == 'false' and workflow.parameters.XXX == 'true' ? 'true' : 'false'}}" # argo/Mutex/{{=inputs.parameters.YYY == 'false' and workflow.parameters.XXX == 'true' ? 'true' : 'false'}}
            echo "{{=inputs.parameters.YYY == 'false' and inputs.parameters.YYY == 'true' ? 'true' : 'false'}}" # argo/Mutex/false
            echo "{{=workflow.parameters.XXX == 'true' and workflow.parameters.XXX == 'false' ? 'true' : 'false'}}" # argo/Mutex/false 

✗ argo submit --log tmp.yaml 
Name:                my-wf-h4p5d
Namespace:           argo
ServiceAccount:      unset (will run with the default ServiceAccount)
Status:              Pending
Created:             Mon Apr 04 07:36:02 -0700 (now)
Progress:            
Parameters:          
  XXX:               true
  YYY:               false

This workflow does not have security context set. You can run your workflow pods more securely by setting it.
Learn more at https://argoproj.github.io/argo-workflows/workflow-pod-security-context/
my-wf-h4p5d: time="2022-04-04T14:36:04.207Z" level=info msg="capturing logs" argo=true
my-wf-h4p5d: true
my-wf-h4p5d: c9f7ac42-fa9d-44fe-8694-7479965a3a32
my-wf-h4p5d: c9f7ac42-fa9d-44fe-8694-7479965a3a32
my-wf-h4p5d: false
my-wf-h4p5d: false
my-wf-h4p5d: true
my-wf-h4p5d: true
my-wf-h4p5d: false
my-wf-h4p5d: false
```

[alexec]

You're correct, and works. I learnt something.

I get a different result to you.

[aaaaahaaaaa]

@alexec What about the original issue?

[alexec]

Could you please re-state the problem?

[aaaaahaaaaa]

@alexec You mean the original post is unclear?

[aaaaahaaaaa]

The gist is, considering the following workflow:

---
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: my-template
  namespace: argo
spec:
  entrypoint: execute

  arguments:
    parameters:
      - name: X
        value: xxx
      - name: Y
        valueFrom:
          configMapKeyRef:
            name: my-config
            key: my-key

  templates:
    - name: execute
      steps:
        - - name: other-template
            templateRef:
              name: other-template
              template: execute
            withParam: "{{workflow.parameters.Y}}"
            arguments:
              parameters:
                - name: my-param
                  value: "{{= sprig.replace('what ever', workflow.parameters.X, item.Y) }}"

Then the following error is raised:

  Warning  WorkflowFailed   3s    workflow-controller  failed to evaluate expression: cannot fetch parameters from <nil> (1:23)
 |  sprig.replace('what ever', workflow.parameters.paramX, item.Y) 
 | ............................^

The problem seems to specifically occur when both worflow.parameters.X and item.Y are used next to each other as parameters of the same function.

And again, the following ARE working:

{{= sprig.replace('what ever', workflow.parameters.X, workflow.parameters.X) }}

{{= sprig.replace('what ever', item.Y, item.Y) }}

Which is why I believe there's clearly a bug here.

[aaaaahaaaaa]

@alexec Would you confirm that looks like a bug to you?

[rkt2spc]

We also have this problem @alexec @aaaaahaaaaa
It's not a problem with sprig. It's a problem with Argo variables expression in general

      steps:
        - - name: loop
            template: script
            arguments:
              parameters:
                - name: ok
                  value: "{{ workflow.parameters.some_param }} : {{ item }}"
                - name: ok2
                  value: "{{= asInt(workflow.parameters.iterations) }}"
                - name: ok3
                  value: "{{= asInt(item) }}"
                - name: not_ok
                  value: "{{= asInt(workflow.parameters.some_param) + asInt(item) }}"
            withSequence:
              count: "5"

[alexec]

It's not a problem with sprig. It's a problem with Argo variables expression in general

Could you please explain more?

[rkt2spc]

@alexec

This should work, but it doesn't. I'm using Argo Workflow v3.3.3

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: debug-
spec:
  arguments:
    parameters:
      - name: message
        value: "hello"

  entrypoint: entrypoint

  templates:
    - name: entrypoint
      steps:
        - - name: broken
            template: speak
            arguments:
              parameters:
                - name: message
                  value: "{{= workflow.parameters.message + ' ' + item }}"
            withSequence:
              count: "3"

    - name: speak
      inputs:
        parameters:
          - name: message
      script:
        image: python:alpine
        command: ["python"]
        source: |-
          print("{{ message }}")

Expectations

{{= workflow.parameters.message + ' ' + item }} should evaluated to hello 0 hello 1 hello 2 for each fanned-out instance of the step broken

Reality

I got

Change {{= workflow.parameters.message + ' ' + item }} to {{ workflow.parameters.message }} {{ item }} and it works as expected.

But in my use-case I need to do some advanced evaluation using both the workflow param and the loop param e.g {{= asInt(workflow.parameters.message) * asInt(item) }}

[gg314]

This might be a long shot, but @aaaaahaaaaa and @rocketspacer, could you try using workflow.parameters['message-param'] instead of workflow.parameters.message-param?

---

edit: this issue is limited to hyphenated parameters

[aaaaahaaaaa]

This might be a long shot, but @aaaaahaaaaa and @rocketspacer, could you try using workflow.parameters['message'] instead of workflow.parameters.message?

Already tried. Same outcome.

[gg314]

That's a shame. I thought I was having the same problem but switching to this syntax fixed it. There is definitely something strange about the parser...

[zjgemi]

I think the problem is that the workflow contexts (workflow.xxx, inputs.xxx, steps.xxx or tasks.xxx) and the loop context (item) cannot be introduced simultaneously for an expression (starts with {{=).

[philippeboyd]

@alexec any news on this? I can replicate the issue with a ternary operator

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: debug-
spec:
  arguments:
    parameters:
      - name: default
        value: "HELLO"
      - name: message
        value: |
          [
            {"a" :1},
            {"a" :1,"b": 2}
          ]

  entrypoint: entrypoint

  templates:
    - name: entrypoint
      steps:
        - - name: broken
            template: speak
            arguments:
              parameters:
                - name: message
                  value: "{{=item.b != nil ? item.b : workflow.parameters.default}}"
            withParam: "{{workflow.parameters.message}}"

    - name: speak
      inputs:
        parameters:
          - name: message
      script:
        image: python:alpine
        command: ["python"]
        source: |-
          print("{{ inputs.parameters.message }}")

However, replacing the value with value: "{{=item.b != nil ? item.b : 'BYE'}}" works. Why can't we reference any workflow.parameters?

[roelarents]

+1. This makes it impossible to select a field in a (json) parameter by loop item. E.g.: {{=sprig.fromJson(input.parameters.somejsonparam)[item]}}

[Arcahub]

@alexec Any thoughts about this issue from the maintainers team ? One year I am still facing this issue and I don't understand why this issue is marked as closed.

[hadams95]

Would my issue be related to this?

I was attempting to map two input parameters to a single output using Helm's dig function.

I have values.yaml for instance:

map:
  x:
    y:  "z"

In my workflowtemplate resource, I have an output parameter:

outputs:
  parameters:
  - name: mapped_value # name of output parameter
    value: "{{dig `{{inputs.parameters.x}}` `{{inputs.parameters.y}}` `<should-be-z>` .Values.map}}"

If I hardcode the parameters passed to the dig function, I get the expected result "z".
Works

"{{dig x y `<should-be-z>` .Values.map}}"
returns
z

However, when I pass the values to the dig function via the input parameters, it always gets the fallback value "<should-be-z>".
Doesn't work

"{{dig `{{inputs.parameters.x}}` `{{inputs.parameters.y}}` `<should-be-z>` .Values.map}}"
returns
<should-be-z>

Even when I verify the input parameter values are exactly as I expect them, I cannot get this helm function to work with them. I utilized a helper function to wrap the dig function and verified that the input parameters are correct and matching the hardcoded values. I even attempted to loop through the map via range and check equivalence between the input parameters and the nested keys however the eq check always returned false. Again, when I hardcoded the exact same values, the eq check returned true. I can't figure out what I'm doing wrong here. It seems I just can't get the input parameters to work with functions. Thinking through, I suppose I cannot expect the produced Workflow manifest to know what to do with a helm template function since Helm would only be responsible for rendering the WorkflowTemplate

Edit/Workaround

I achieved my desired functionality by adding a jq init container which retrieves the mapped value via a JSON map mounted into the container. The jq init container simply outputs the mapped result to an arbitrary file in a volume shared between both containers. The main container in the workflow then gets that value from the shared volume.

[nishant-jain-94]

+1. I am too facing this issue. I am not able to use the loop context and the workflow context together.