Workflow ignoring nodeSelector and tolerations when defined in WorkflowTemplate

[nice-pink]

Pre-requisites

• I have double-checked my configuration
• I can confirm the issue exists when I tested with :latest
• I have searched existing issues and could not find a match for this bug
• I'd like to contribute the fix myself (see contributing guide)

What happened/what did you expect to happen?

I defined a WorkflowTemplate containing nodeSelector and tolerations. When I manually submit the WorkflowTemplate from the UI both fields are ignored and the workflow is running on any available node in the cluster.

Version

v3.5.4

Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: benchmarks
spec:
  entrypoint: main
  serviceAccountName: workflow
  templates:
  - dag:
      tasks:
      - arguments:
          parameters:
          - name: msg
            value: 'hello'
        name: benchmark
        template: benchmark
    name: main
    nodeSelector:
      pool: workflows
    tolerations:
    - key: pool
      operator: Equal
      value: workflows
  - inputs:
      parameters:
      - name: msg
    name: benchmark
    script:
      command:
      - python
      image: python:latest
      source: |
        print("{{inputs.parameters.msg}}")

Logs from the workflow controller

kubectl logs -n argo deploy/workflow-controller | grep ${workflow}

Logs from in your workflow's wait container

kubectl logs -n argo -c wait -l workflows.argoproj.io/workflow=${workflow},workflow.argoproj.io/phase!=Succeeded

[tooptoop4]

@nice-pink have u tried v3.4.11?

[nice-pink]

No I haven't as we're already on v3.5.4.

[shuangkun]

It seems that we have not implemented the child template's inheritance of the parent template nodeSelector.Currently only workflow-level and node-level nodeSelector are supported.