Helm gcs gateway support

[dhughesni]

Description

• These changes have been migrated from the previous chart: https://github.com/helm/charts/tree/master/stable/minio
• Added gcs support for gateway mode.
• Added new gcs block under the gateway key to house the gcs specific variables.
• The gateway-deployment template now sets the env var: GOOGLE_APPLICATION_CREDENTIALS as a path to the gcs_key.json
• The gcs_key.json can be added to the minio secret if an existingSecret is not specified.

Motivation and Context

• The current chart does not enable gateway mode to be set as gcs. We need the ability to deploy minio as a gcs gateway via helm.
• Minio Slack Reference: https://minio.slack.com/archives/C3NDUB8UA/p1637607661036000

How to test this PR?

$ cd helm/minio/

modify values for gcs gateway deployment

$ git diff
diff --git a/helm/minio/values.yaml b/helm/minio/values.yaml
index 604beb39e..50c2ac171 100644
--- a/helm/minio/values.yaml
+++ b/helm/minio/values.yaml
@@ -32,7 +32,7 @@ mcImage:
 ## Distributed MinIO ref: https://docs.minio.io/docs/distributed-minio-quickstart-guide
 ## NAS Gateway MinIO ref: https://docs.min.io/docs/minio-gateway-for-nas.html
 ## GCS Gateway MinIO ref: https://docs.min.io/docs/minio-gateway-for-gcs.html
-mode: distributed
+mode: gateway
 
 ## NOTE: currently only "nas,gcs" gateways are supported.
 
@@ -106,11 +106,11 @@ pools: 1
 
 # Deploy if 'mode == gateway' - 4 replicas.
 gateway:
-  type: "nas" # currently only "nas,gcs" are supported.
-  replicas: 4
+  type: "gcs" # currently only "nas,gcs" are supported.
+  replicas: 1
   gcs:
-    gcsKeyJson: "" # credential json file of service account key (not required if using existingSecret)
-    projectId: "" # Google cloud project id required
+    gcsKeyJson: "<GCS_JSON_SERVICE_ACCOUNT>" # credential json file of service account key
+    projectId: "<GCS_PROJECT_ID>" # Google cloud project id
 
 ## TLS Settings for MinIO
 tls:
@@ -130,7 +130,7 @@ trustedCertsSecret: ""
 ## ref: http://kubernetes.io/docs/user-guide/persistent-volumes/
 ##
 persistence:
-  enabled: true
+  enabled: false

debug install as dry-run to check entrypoint is configured correctly

$ helm install miniogcs . --debug --dry-run | grep "minio gateway gcs"
...
(standard input):496:            - "/usr/bin/docker-entrypoint.sh minio gateway gcs dr-exploration-team -S /etc/minio/certs/ --address :9000 --console-address :9001 "

test chart install

$ helm install minio .
NAME: minio
LAST DEPLOYED: Thu Dec  2 15:57:07 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Minio can be accessed via port 9000 on the following DNS name from within your cluster:
minio.default.svc.cluster.local
To access Minio from localhost, run the below commands:
  1. export POD_NAME=$(kubectl get pods --namespace default -l "release=minio" -o jsonpath="{.items[0].metadata.name}")
  2. kubectl port-forward $POD_NAME 9000 --namespace default
Read more about port forwarding here: http://kubernetes.io/docs/user-guide/kubectl/kubectl_port-forward/
You can now access Minio server on http://localhost:9000. Follow the below steps to connect to Minio server with mc client:
  1. Download the Minio mc client - https://docs.minio.io/docs/minio-client-quickstart-guide
  2. export ACCESS_KEY=$(kubectl get secret minio -o jsonpath="{.data.rootUser}" | base64 --decode) 
  3. export SECRET_KEY=$(kubectl get secret minio -o jsonpath="{.data.rootPassword}" | base64 --decode)
  4. mc alias set minio-local http://localhost:9000 "$ACCESS_KEY" "$SECRET_KEY" --api s3v4
  5. mc ls minio-local
Alternately, you can use your browser or the Minio SDK to access the server - https://docs.minio.io/categories/17

$ export POD_NAME=$(kubectl get pods --namespace default -l "release=minio" -o jsonpath="{.items[0].metadata.name}")
$ kubectl port-forward $POD_NAME 9000 --namespace default &
[1] 26269
Forwarding from 127.0.0.1:9000 -> 9000
Forwarding from [::1]:9000 -> 9000
$ export ACCESS_KEY=$(kubectl get secret minio -o jsonpath="{.data.rootUser}" | base64 --decode) 
$ export SECRET_KEY=$(kubectl get secret minio -o jsonpath="{.data.rootPassword}" | base64 --decode)
$ mc alias set minio-local http://localhost:9000 "$ACCESS_KEY" "$SECRET_KEY" --api s3v4
mc: Configuration written to `/Users/daryl.hughes/.mc/config.json`. Please update your access credentials.
mc: Successfully created `/Users/daryl.hughes/.mc/share`.
mc: Initialized share uploads `/Users/daryl.hughes/.mc/share/uploads.json` file.
mc: Initialized share downloads `/Users/daryl.hughes/.mc/share/downloads.json` file.
Added `minio-local` successfully.
$ mc ls minio-local
Handling connection for 9000
... Testing validated: GCS object store buckets are listed confirming minio is configured as gcs gateway

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Optimization (provides speedup with no functional changes)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• Fixes a regression (If yes, please add commit-id or PR # here)
• Documentation updated
• Unit tests added/updated

[harshavardhana]

Is this still draft @dhughesni ?

[dhughesni]

@harshavardhana Yes still a draft atm.
I plan to do some final testing before the end of this week - at that point I will convert to PR ready for review 👍

[dhughesni]

@harshavardhana PR is ready for review - please let me know if there are any further actions required from me.

[minio-trusted]

Mint Automation

Test	Result
mint-large-bucket.sh	✔️
mint-fs.sh	✔️
mint-gateway-s3.sh	✔️
mint-erasure.sh	✔️
mint-dist-erasure.sh	✔️
mint-gateway-nas.sh	✔️
mint-compress-encrypt-dist-erasure.sh	✔️
mint-pools.sh	✔️
Deleting image on docker hub
Deleting image locally