volumes in WorkflowTemplates are not carried over to Workflows

[foobarbecue]

Checklist:

• I've included the version.
• I've included reproduction steps.
• I've included the workflow YAML.
• I've included the logs.

What happened:

This Workflow suceeds, and writes a file test.txt to the host:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: voltest-
  namespace: default
spec:
  entrypoint: test-vols
  volumes:
    - name: nac-volume
      hostPath:
        path: /c/testdir
        type: DirectoryOrCreate

  templates:
    - name: test-vols
      container:
        image: ubuntu
        volumeMounts:
          - name: nac-volume
            mountPath: /data
        command: [sh, -c]
        args: ["echo test > /data/test.txt"]

However, if you convert it to a WorkflowTemplate and submit a Workflow based on it, then it fails with
volume 'nac-volume' not found in workflow spec

Here's the WorkflowTemplate version:

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: voltesttmpl
  generateName: voltest-
  namespace: default
spec:
  entrypoint: test-vols
  volumes:
    - name: nac-volume
      hostPath:
        path: /c/testdir
        type: DirectoryOrCreate

  templates:
    - name: test-vols
      container:
        image: ubuntu
        volumeMounts:
          - name: nac-volume
            mountPath: /data
        command: [sh, -c]
        args: ["echo test > /data/test.txt"]

What you expected to happen:

I expected the volumes to be carried over to the Template from the WorkflowTemplate.

How to reproduce it (as minimally and precisely as possible):

See "what happened" above.

Environment:

• Argo version:

$ argo version
argo: v2.9.0-rc4
  BuildDate: 2020-06-26T21:54:40Z
  GitCommit: 5b109bcb9257653ecbae46e6315c8d65842de58a
  GitTreeState: clean
  GitTag: v2.9.0-rc4
  GoVersion: go1.13.4
  Compiler: gc
  Platform: linux/amd64

• Kubernetes version :

$ kubectl version -o yaml
clientVersion:
  buildDate: "2019-11-13T11:23:11Z"
  compiler: gc
  gitCommit: b3cbbae08ec52a7fc73d334838e18d17e8512749
  gitTreeState: clean
  gitVersion: v1.16.3
  goVersion: go1.12.12
  major: "1"
  minor: "16"
  platform: linux/amd64
serverVersion:
  buildDate: "2020-01-15T08:18:29Z"
  compiler: gc
  gitCommit: e7f962ba86f4ce7033828210ca3556393c377bcc
  gitTreeState: clean
  gitVersion: v1.16.6-beta.0
  goVersion: go1.13.5
  major: "1"
  minor: 16+
  platform: linux/amd64

Message from the maintainers:

If you are impacted by this bug please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

[alexec]

@foobarbecue do you know if this is a new problem?

[foobarbecue]

@alexec I don't know -- this was my first time trying volumes with WorkflowTemplates. So it's probably not new. BTW I also tried https://github.com/argoproj/argo/blob/master/examples/volumes-emptydir.yaml as a WorkflowTemplate with the same result.

Probably time for me to learn to use PVCs anyway ;-)

Edit: PVCs don't work either

[foobarbecue]

I'm working on this now:

• Adding the missing properties from WorkflowSpec to WorkflowTemplateSpec in ui/src/models
• Correcting the getWorkflow function in ui/src/app/workflow-templates/components/workflow-template-details/workflow-template-details.tsx so that it copies those properties over

Presumably there needs to be a server-side fix as well, for when you do argo submit --from workflowtemplate/mywft

I am more and more convinced that argo shot itself in the foot when it created workflowtemplates instead of adding features to templates :-p

[alexec]

@sarabala1979 would you like to own this please?

[foobarbecue]

Yep, can do. I expect to have a PR today. Edit: Oh lol didn't read properly. @sarabala1979 can own it, for sure. I'll submit a PR in a bit though.

[foobarbecue]

Oh, sorry I was working on an old fork... there is no more WorkflowTemplateSpec in ui/src/models, it just references TemplateSpec... so that's a good thing. So it's just a matter of fixing the getWorkflow function now (and then the CLI fixes).

[foobarbecue]

Draft PR: #3441

[alexec]

I this #3436 fixes this too.

[foobarbecue]

Ah, cool, I'll try out #3437 and close this if it works

[foobarbecue]

I just tried it, at 81cba83 and get volume 'nac-volume' not found in workflow spec . I'm just gonna wait a while since it seems workflowtemplates are still being enhanced to support all the fields that workflows support.

[alexec]

@foobarbecue - this needs two fixes. One to remove the panic that obscures the error (this was a pre-existing bug) an another to fix the actual bug #3441.

[foobarbecue]

Ah, ok. When we make a WF from a WFT, and the WF is referring to the WFT using workflowTemplateRef, do we want the volumes and stuff to be looked up on the WFT through the reference? Or do we want to copy them to the WF? My #3441 copies them to the WF, which I'm now thinking is the wrong way of doing it.

I guess I'm going to have to study how workflowTemplateRefs are actually implemented.

[alexec]

Fixed in v2.9

[foobarbecue]

Just to confirm, I'm using this now and it works well!