Proposal: argo cp command to download artifacts

[jessesuen]

This proposal is for an argo CLI command to download artifacts from a workflow step. Given a workflow result like:

$ argo get artifact-passing-jg2k8 -o wide
Name:             artifact-passing-jg2k8
Namespace:        default
ServiceAccount:   default
Status:           Succeeded
Created:          Thu Jan 18 15:02:42 -0800 (9 hours ago)
Started:          Thu Jan 18 15:02:42 -0800 (9 hours ago)
Finished:         Thu Jan 18 15:03:31 -0800 (9 hours ago)
Duration:         49 seconds

STEP                       PODNAME                            DURATION  ARTIFACTS  MESSAGE
 ✔ artifact-passing-jg2k8
 ├---✔ generate-artifact   artifact-passing-jg2k8-2089725168  25s       hello-art
 └---✔ consume-artifact    artifact-passing-jg2k8-2707584000  23s

One could do something like:

$ argo cp artifact-passing-jg2k8-2089725168:hello-art /tmp/

The workflow YAML already has most of the information required to download the artifact (e.g. bucket name and key) as seen in the node status:

    artifact-passing-jg2k8-2089725168:
      finishedAt: 2018-01-18T23:03:07Z
      id: artifact-passing-jg2k8-2089725168
      name: artifact-passing-jg2k8[0].generate-artifact
      outputs:
        artifacts:
        - name: hello-art
          path: /tmp/hello_world.txt
          s3:
            accessKeySecret:
              key: accesskey
              name: argo-artifacts-minio-user
            bucket: my-bucket
            endpoint: argo-artifacts-minio-svc.default:9000
            insecure: true
            key: artifact-passing-jg2k8/artifact-passing-jg2k8-2089725168/hello-art.tgz
            secretKeySecret:
              key: secretkey
              name: argo-artifacts-minio-user

The issues here are that:

1. the artifact repository may not directly accessible from the client (e.g. internal minio server)
2. the secrets may not be accessible

However, we could at least support the use case for GCS and AWS, where the repository will be directly accessible. Regarding bucket secrets, we can either lookup the secrets using the k8s go-client, or rely on the .aws config/credentials in the user's home directory for the download.

[jessesuen]

Would be possible to do with a proper API server.

[alexec]

You can now curl artifacts.