feat: Native Google Cloud Storage support for artifact. Closes #1911

[whynowy]

Closes #1911 and #1067

This commit implements the all the features of using GCS as artifact repository with Argo.

• Use GCS as artifact input, output, can be either a file or directory
• Configure GCS as default artifact repository in Argo configmap
• Hardwire GCS artifact, file or directory

A GCS artifact config looks like following:

        artifacts:
          - name: my-art
            path: /my-artifact
            gcs:
              bucket: my-bucket-name
              # key could be either a file or a directory.
              key: path/in/bucket
              # serviceAccountKeySecret is a secret selector.
              # It references the k8s secret named 'my-gcs-credentials'.
              # This secret is expected to have have the key 'serviceAccountKey',
              # containing the base64 encoded Google Cloud Service Account Key (json)
              # to access the bucket.
              #
              # If it's running on GKE and Workload Identity is configured,
              # serviceAccountKeySecret is not needed.
              serviceAccountKeySecret:
                name: my-gcs-credentials
                key: serviceAccountKey

Example default GCS artifact configuration in the Configmap:

data:
  config: |
    artifactRepository:
      gcs:
        bucket: my-bucket
        #keyFormat is optional, it could reference workflow variables, such as "{{workflow.name}}/{{pod.name}}"
        keyFormat: prefix/in/bucket
        # serviceAccountKeySecret is not needed if Workload Identity is configured
        serviceAccountKeySecret:
          name: my-gcs-credentials
          key: serviceAccountKey

Checklist:

• Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this is a chore.
• The title of the PR is (a) conventional, (b) states what changed, and (c) suffixes the related issues number. E.g. "fix(controller): Updates such and such. Fixes #1234".
• I have written unit and/or e2e tests for my change. PRs without these are unlike to be merged.
• Optional. I've added My organization is added to the USERS.md.
• I've signed the CLA and required builds are green.

[whynowy]

A separate PR to address the comment #1067 (comment), which is about Workload Identity.

[codecov]

Codecov Report

❗ No coverage uploaded for pull request base (master@999b1e1). Click here to learn what that means.
The diff coverage is 0%.

@@            Coverage Diff            @@
##             master    #2484   +/-   ##
=========================================
  Coverage          ?   11.21%           
=========================================
  Files             ?       75           
  Lines             ?    31690           
  Branches          ?        0           
=========================================
  Hits              ?     3555           
  Misses            ?    27657           
  Partials          ?      478

Impacted Files	Coverage Δ
config/config.go	33.33% <ø> (ø)
pkg/apis/workflow/v1alpha1/generated.pb.go	0.46% <ø> (ø)
...kg/apis/workflow/v1alpha1/zz_generated.deepcopy.go	0% <0%> (ø)
pkg/apis/workflow/v1alpha1/workflow_types.go	11.45% <0%> (ø)
workflow/controller/workflowpod.go	70.66% <0%> (ø)
workflow/executor/executor.go	4.45% <0%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 999b1e1...cf89e04. Read the comment docs.

[whynowy]

A separate PR to address the comment #1067 (comment), which is about Workload Identity.

I did it in this PR, serviceAccountKeySecret is optional, if it's absent, it assumes workload Identity is configured, and will use a default client, which will call metadata API underneath to get a token.

[simster7]

This LGTM. Having tests would be ideal, but given the nature of the PR they might be difficult to produce. However, since this PR is mostly orthogonal, I think it should be fine.

I would suggest also getting @alexec or @sarabala1979 to take a look before merging.

[whynowy]

This LGTM. Having tests would be ideal, but given the nature of the PR they might be difficult to produce. However, since this PR is mostly orthogonal, I think it should be fine.

I would suggest also getting @alexec or @sarabala1979 to take a look before merging.

Thanks @simster7 !

[whynowy]

@alexec @sarabala1979 , do you have any comments?

[alexec]

I trust Simon as much as myself on this one.