UserContainer swagger definition not matching command line client

[cburbridge]

What happened:

The API swagger file for release 2.7 (https://github.com/argoproj/argo/blob/release-2.7/api/openapi-spec/swagger.json) defines a UserContainer as a

"io.argoproj.workflow.v1alpha1.UserContainer": {
      "description": "UserContainer is a container specified by a user.",
      "type": "object",
      "properties": {
        "container": {
          "$ref": "#/definitions/io.k8s.api.core.v1.Container"
        },
        "mirrorVolumeMounts": {
          "type": "boolean",
          "format": "boolean",
          "title": "MirrorVolumeMounts will mount the same volumes specified in the main container\nto the container (including artifacts), at the same mountPaths. This enables\ndind daemon to partially see the same filesystem as the main container in\norder to use features such as docker volume binding"
        }
      }
    },

and the sidecars property of a Template as an array of UserContainers.

In the sidecar example at https://github.com/argoproj/argo/blob/master/examples/sidecar.yaml, the YAML for the sidecar property looks like:

    sidecars:
    - name: influxdb
      image: influxdb:1.2

whereas the swagger file suggests it should be generated as:

    sidecars:
    - container:
        name: influxdb
        image: influxdb:1.2

However, the argo command line client will not accept this format:

argo submit sidecars_wf.yaml --serviceaccount argo --instanceid "27"
2020/04/24 15:31:46 Failed to parse workflow: error unmarshaling JSON: while decoding JSON: json: unknown field "container"

What you expected to happen:
The argo client (version 2.7.5) to accept the UserContainer in the form from the 2.7 swagger file.

It looks like the example and command line client still follow the swagger file from release 2.5.

Environment:

• Argo version:

argo: v2.7.5
  BuildDate: 2020-04-21T00:55:01Z
  GitCommit: ede163e1af83cfce29b519038be8127664421329
  GitTreeState: clean
  GitTag: v2.7.5
  GoVersion: go1.13
  Compiler: gc
  Platform: linux/amd64

• Kubernetes version :

clientVersion:
  buildDate: "2020-02-11T18:14:22Z"
  compiler: gc
  gitCommit: 06ad960bfd03b39c8310aaf92d1e7c12ce618213
  gitTreeState: clean
  gitVersion: v1.17.3
  goVersion: go1.13.6
  major: "1"
  minor: "17"
  platform: linux/amd64
serverVersion:
  buildDate: "2020-02-07T01:31:02Z"
  compiler: gc
  gitCommit: 502bfb383169b124d87848f89e17a04b9fc1f6f0
  gitTreeState: clean
  gitVersion: v1.14.9-eks-502bfb
  goVersion: go1.12.12
  major: "1"
  minor: 14+
  platform: linux/amd64

---

Message from the maintainers:

If you are impacted by this bug please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

[alexec]

https://github.com/argoproj/argo/blob/master/docs/fields.md#usercontainer

[alexec]

You're quite correct - I wonder if this is a problem with Golang inline.

[alexec]

Yes. There are multiple instances, cf S3Bucket.

[alexec]

Ok. I've investigated - and I believe this is correct - the Swagger reflects the API, not the YAML. These are different, that is to be expected.

[cburbridge]

Thank you for investigating this.

Instead of pushing swagger-spec generated YAML through the argoclient, I have tried calling the API directly with a WorkflowCreateRequest request object created following the swagger. I still get the same issue around the use of a 2.6+ UserContainer instead of the older 2.5 UserContainer:

In [1]: import requests                                                                                                                                                     
In [2]: req = \ 
   ...: { "instanceID": "27", 
   ...:   "namespace": "dev", 
   ...:   "workflow": { 
   ...:     "metadata": { 
   ...:       "generateName": "sidcar-nginx-" 
   ...:     }, 
   ...:     "spec": { 
   ...:       "entrypoint": "sidecar-nginx-example", 
   ...:       "templates": [ 
   ...:         { 
   ...:           "container": { 
   ...:             "args": [ 
   ...:               "until `curl -G 'http://127.0.0.1/' >& /tmp/out`; do echo sleep && sleep 1; done && cat /tmp/out" 
   ...:             ], 
   ...:             "command": [ 
   ...:               "sh", 
   ...:               "-c" 
   ...:             ], 
   ...:             "image": "appropriate/curl" 
   ...:           }, 
   ...:           "name": "sidecar-nginx-example", 
   ...:           "sidecars": [ 
   ...:             { 
   ...:               "container": { 
   ...:                 "image": "nginx:1.13", 
   ...:                 "name": "nginx" 
   ...:               } 
   ...:             } 
   ...:           ] 
   ...:         } 
   ...:       ] 
   ...:     } 
   ...:   } 
   ...: }                                                                                                                                                                   

In [3]: requests.post("https://argo27.dev.pipeline.five.ai/api/v1/workflows/dev", json=req).json()                                                                      
Out[3]: 
{'metadata': {'name': 'sidcar-nginx-mgljm',
  'generateName': 'sidcar-nginx-',
  'namespace': 'dev',
  'selfLink': '/apis/argoproj.io/v1alpha1/namespaces/dev/workflows/sidcar-nginx-mgljm',
  'uid': '28cc5e7e-8789-11ea-83cc-0a99a6e6d31c',
  'resourceVersion': '111829529',
  'generation': 1,
  'creationTimestamp': '2020-04-26T06:42:55Z',
  'labels': {'workflows.argoproj.io/controller-instanceid': '27'}},
 'spec': {'templates': [{'name': 'sidecar-nginx-example',
    'arguments': {},
    'inputs': {},
    'outputs': {},
    'metadata': {},
    'container': {'name': '',
     'image': 'appropriate/curl',
     'command': ['sh', '-c'],
     'args': ["until `curl -G 'http://127.0.0.1/' >& /tmp/out`; do echo sleep && sleep 1; done && cat /tmp/out"],
     'resources': {}},
    'sidecars': [{'name': '', 'resources': {}}]}],
  'entrypoint': 'sidecar-nginx-example',
  'arguments': {}},
 'status': {'startedAt': None, 'finishedAt': None}}

In [4]: requests.get("https://argo27.dev.pipeline.five.ai/api/v1/workflows/dev/sidcar-nginx-mgljm").json()                                                              
Out[4]: 
{'metadata': {'name': 'sidcar-nginx-mgljm',
  'generateName': 'sidcar-nginx-',
  'namespace': 'dev',
  'selfLink': '/apis/argoproj.io/v1alpha1/namespaces/dev/workflows/sidcar-nginx-mgljm',
  'uid': '28cc5e7e-8789-11ea-83cc-0a99a6e6d31c',
  'resourceVersion': '111829537',
  'generation': 3,
  'creationTimestamp': '2020-04-26T06:42:55Z',
  'labels': {'workflows.argoproj.io/completed': 'true',
   'workflows.argoproj.io/controller-instanceid': '27',
   'workflows.argoproj.io/phase': 'Error'}},
 'spec': {'templates': [{'name': 'sidecar-nginx-example',
    'arguments': {},
    'inputs': {},
    'outputs': {},
    'metadata': {},
    'container': {'name': '',
     'image': 'appropriate/curl',
     'command': ['sh', '-c'],
     'args': ["until `curl -G 'http://127.0.0.1/' >& /tmp/out`; do echo sleep && sleep 1; done && cat /tmp/out"],
     'resources': {}},
    'sidecars': [{'name': '', 'resources': {}}]}],
  'entrypoint': 'sidecar-nginx-example',
  'arguments': {}},
 'status': {'phase': 'Error',
  'startedAt': '2020-04-26T06:42:55Z',
  'finishedAt': '2020-04-26T06:42:56Z',
  'message': 'Pod "sidcar-nginx-mgljm" is invalid: [spec.containers[2].name: Required value, spec.containers[2].image: Required value]',
  'nodes': {'sidcar-nginx-mgljm': {'id': 'sidcar-nginx-mgljm',
    'name': 'sidcar-nginx-mgljm',
    'displayName': 'sidcar-nginx-mgljm',
    'type': 'Pod',
    'templateName': 'sidecar-nginx-example',
    'phase': 'Error',
    'message': 'Pod "sidcar-nginx-mgljm" is invalid: [spec.containers[2].name: Required value, spec.containers[2].image: Required value]',
    'startedAt': '2020-04-26T06:42:55Z',
    'finishedAt': '2020-04-26T06:42:55Z'}},
  'conditions': [{'type': 'Completed', 'status': 'True'}]}}

'Pod "sidcar-nginx-mgljm" is invalid: [spec.containers[2].name: Required value, spec.containers[2].image: Required value]

happens because the API appears to expect the containers field to be a list of objects with Container attributes at the top level, not subordinated under the container attribute. Without doing this the workflow parses and gets run.

[alexec]

Hmm. Needs further investigate.

[alexec]

find pkg -name '*_types.go'|xargs grep -n 'type\|,inline'|grep -v TypeMeta|grep -B1 ',inline'
pkg/apis/workflow/v1alpha1/workflow_types.go:603:type Artifact struct {
pkg/apis/workflow/v1alpha1/workflow_types.go:618:	ArtifactLocation `json:",inline" protobuf:"bytes,5,opt,name=artifactLocation"`
--
pkg/apis/workflow/v1alpha1/workflow_types.go:857:type UserContainer struct {
pkg/apis/workflow/v1alpha1/workflow_types.go:858:	apiv1.Container `json:",inline" protobuf:"bytes,1,opt,name=container"`
--
pkg/apis/workflow/v1alpha1/workflow_types.go:1266:type S3Artifact struct {
pkg/apis/workflow/v1alpha1/workflow_types.go:1267:	S3Bucket `json:",inline" protobuf:"bytes,1,opt,name=s3Bucket"`
--
pkg/apis/workflow/v1alpha1/workflow_types.go:1319:type ArtifactoryArtifact struct {
pkg/apis/workflow/v1alpha1/workflow_types.go:1322:	ArtifactoryAuth `json:",inline" protobuf:"bytes,2,opt,name=artifactoryAuth"`
--
pkg/apis/workflow/v1alpha1/workflow_types.go:1334:type HDFSArtifact struct {
pkg/apis/workflow/v1alpha1/workflow_types.go:1335:	HDFSConfig `json:",inline" protobuf:"bytes,1,opt,name=hDFSConfig"`
--
pkg/apis/workflow/v1alpha1/workflow_types.go:1349:type HDFSConfig struct {
pkg/apis/workflow/v1alpha1/workflow_types.go:1350:	HDFSKrbConfig `json:",inline" protobuf:"bytes,1,opt,name=hDFSKrbConfig"`
--
pkg/apis/workflow/v1alpha1/workflow_types.go:1418:type GCSArtifact struct {
pkg/apis/workflow/v1alpha1/workflow_types.go:1419:	GCSBucket `json:",inline" protobuf:"bytes,1,opt,name=gCSBucket"`
--
pkg/apis/workflow/v1alpha1/workflow_types.go:1445:type OSSArtifact struct {
pkg/apis/workflow/v1alpha1/workflow_types.go:1446:	OSSBucket `json:",inline" protobuf:"bytes,1,opt,name=oSSBucket"`
--
pkg/apis/workflow/v1alpha1/workflow_types.go:1463:type ScriptTemplate struct {
pkg/apis/workflow/v1alpha1/workflow_types.go:1464:	apiv1.Container `json:",inline" protobuf:"bytes,1,opt,name=container"`
--
pkg/apis/workflow/v1alpha1/workflow_template_types.go:44:type WorkflowTemplateSpec struct {
pkg/apis/workflow/v1alpha1/workflow_template_types.go:45:	WorkflowSpec `json:",inline" protobuf:"bytes,1,opt,name=workflowSpec"`