stable/concourse: separate worker, web deployments (helm#12920)

* stable/concourse: separate worker, web deployments - Added .Values.Web.enabled and .Values.worker.enabled (with default to true). - if only .Values.web.enable is enabled: only web resources are going to be created, as well as secrets namespace. - if only .Values.worker.enable is enabled: only worker resources are going to be created. - moved the worker specific and web specific secrets each to a seprate file and secrets object. - added .Values.concourse.worker.tsa.port and utilised .Values.concourse.worker.host in order to alow the user to set the `CONCOURSE_TSA_HOST` easily. - removed the template `concourse.concourse.fullname` as it is not used anymore. - bumped the chart version to 6.0.0, as this adds new ways to use the chart. - added the new variables to the README doc. for the issue: helm#11280 Signed-off-by: Taylor Silva <tsilva@pivotal.io> Co-authored-by: Bishoy Youssef <byoussef@pivotal.io> Signed-off-by: Taylor Silva <tsilva@pivotal.io> * [stable/concourse] bumping chart version and updating the Readme Signed-off-by: Taylor Silva <tsilva@pivotal.io> Co-authored-by: Bishoy Youssef <byoussef@pivotal.io> * Adds a required check yaml for having either the web or worker enabled Signed-off-by: Bishoy Youssef <byoussef@pivotal.io> * Adds required for concourse.worker.tsa in case of worker only deployments Signed-off-by: Bishoy Youssef <byoussef@pivotal.io> * updates readme Signed-off-by: Bishoy Youssef <byoussef@pivotal.io> * adds changelog file Signed-off-by: Bishoy Youssef <byoussef@pivotal.io> * Apply suggestions from code review Signed-off-by: Bishoy Youssef <byoussef@pivotal.io>
dermorz · Apr 26, 2019 · f791ede · f791ede
1 parent c14f02e
commit f791ede
Show file tree

Hide file tree

Showing 21 changed files with 129 additions and 49 deletions.
diff --git a/stable/concourse/CHANGELOG.md b/stable/concourse/CHANGELOG.md
@@ -0,0 +1,4 @@
+## v6.0.0:
+
+- added the ability to create worker only and web-only deployments using `web.enabled` and `worker.enabled`
+- **[breaking]** worker and web secrets are now separated into 2 different templates, `worker-secrets.yaml` and `web-secrets.yaml`. Users bringing their own secrets will have to split them into 2 different k8s objects.
diff --git a/stable/concourse/Chart.yaml b/stable/concourse/Chart.yaml
@@ -1,5 +1,5 @@
 name: concourse
-version: 5.2.3
+version: 6.0.0
 appVersion: 5.1.0
 description: Concourse is a simple and scalable CI system.
 icon: https://avatars1.githubusercontent.com/u/7809479

diff --git a/stable/concourse/README.md b/stable/concourse/README.md
@@ -154,6 +154,7 @@ The following table lists the configurable parameters of the Concourse chart and
 | `web.additionalVolumes` | Volumes to be added to the web pods | `nil` |
 | `web.annotations`| Concourse Web deployment annotations | `nil` |
 | `web.authSecretsPath` | Specify the mount directory of the web auth secrets | `/concourse-auth` |
+| `web.enabled` | Enable or disable the web component | `true` |
 | `web.env` | Configure additional environment variables for the web containers | `[]` |
 | `web.ingress.annotations` | Concourse Web Ingress annotations | `{}` |
 | `web.ingress.enabled` | Enable Concourse Web Ingress | `false` |
@@ -193,6 +194,7 @@ The following table lists the configurable parameters of the Concourse chart and
 | `worker.annotations` | Annotations to be added to the worker pods | `{}` |
 | `worker.cleanUpWorkDirOnStart` | Removes any previous state created in `concourse.worker.workDir` | `true` |
 | `worker.emptyDirSize` | When persistance is disabled this value will be used to limit the emptyDir volume size | `nil` |
+| `worker.enabled` | Enable or disable the worker component. You should set postgres.enabled=false in order not to get an unnecessary Postgres chart deployed | `true` |
 | `worker.env` | Configure additional environment variables for the worker container(s) | `[]` |
 | `worker.hardAntiAffinity` | Should the workers be forced (as opposed to preferred) to be on different nodes? | `false` |
 | `worker.keySecretsPath` | Specify the mount directory of the worker keys secrets | `/concourse-keys` |
@@ -262,7 +264,13 @@ rm session-signing-key.pub
 printf "%s:%s" "concourse" "$(openssl rand -base64 24)" > local-users
 ```
 
-You'll also need to create/copy secret values for optional features. See [templates/secrets.yaml](templates/secrets.yaml) for possible values.
+All the worker-specific secrets, namely, `workerKey`, `workerKeyPub`, `hostKeyPub` are to be added to a separate Kubernetes secrets object with the name [release name]-worker.
+
+All other secrets are to be added to a secrets object with the name `[release name]-web`.
+
+For the time being, the secret `workerKeyPub` is to be added to both the worker and the web secret objects, until investigated within issue #13019.
+
+You'll also need to create/copy secret values for optional features. See [templates/web-secrets.yaml](templates/web-secrets.yaml) and [templates/web-secrets.yaml](templates/web-secrets.yaml)  for possible values.
 
 In the example below, we are not using the [PostgreSQL](#postgresql) chart dependency, and so we must set `postgresql-user` and `postgresql-password` secrets.
 

diff --git a/stable/concourse/templates/_helpers.tpl b/stable/concourse/templates/_helpers.tpl
@@ -7,19 +7,18 @@ Expand the name of the chart.
 {{- end -}}
 
 {{/*
-Create a default fully qualified concourse name.
+Create a default fully qualified web node(s) name.
 We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
 */}}
-{{- define "concourse.concourse.fullname" -}}
-{{- $name := default .Chart.Name .Values.nameOverride -}}
-{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
-{{- end -}}
-
 {{- define "concourse.web.fullname" -}}
 {{- $name := default "web" .Values.web.nameOverride -}}
 {{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
 {{- end -}}
 
+{{/*
+Create a default fully qualified worker node(s) name.
+We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
+*/}}
 {{- define "concourse.worker.fullname" -}}
 {{- $name := default "worker" .Values.worker.nameOverride -}}
 {{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}

diff --git a/stable/concourse/templates/namespace.yaml b/stable/concourse/templates/namespace.yaml
@@ -1,3 +1,4 @@
+{{- if .Values.web.enabled -}}
 {{- if and .Values.concourse.web.kubernetes.enabled .Values.concourse.web.kubernetes.createTeamNamespaces -}}
 {{- range .Values.concourse.web.kubernetes.teams }}
 ---
@@ -10,9 +11,10 @@ metadata:
 {{- end }}
   name: {{ template "concourse.namespacePrefix" $ }}{{ . }}
   labels:
-    app: {{ template "concourse.concourse.fullname" $ }}
+    app: {{ template "concourse.web.fullname" $ }}
     chart: "{{ $.Chart.Name }}-{{ $.Chart.Version }}"
     release: "{{ $.Release.Name }}"
     heritage: "{{ $.Release.Service }}"
 {{- end }}
+{{- end }}
 {{- end -}}
diff --git a/stable/concourse/templates/required-check.yaml b/stable/concourse/templates/required-check.yaml
@@ -0,0 +1,3 @@
+{{ if not (or .Values.web.enabled .Values.worker.enabled) }}
+{{- required "Must set either web.enabled or worker.enabled to create a concourse deployment" "" }}
+{{ end }}
diff --git a/stable/concourse/templates/web-deployment.yaml b/stable/concourse/templates/web-deployment.yaml
@@ -1,3 +1,4 @@
+{{- if .Values.web.enabled -}}
 apiVersion: extensions/v1beta1
 kind: Deployment
 metadata:
@@ -92,12 +93,12 @@ spec:
             - name: CONCOURSE_BITBUCKET_CLOUD_CLIENT_ID
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: bitbucket-cloud-client-id
             - name: CONCOURSE_BITBUCKET_CLOUD_CLIENT_SECRET
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: bitbucket-cloud-client-secret
             {{- end }}
             {{- if .Values.concourse.web.logLevel }}
@@ -116,7 +117,7 @@ spec:
             - name: CONCOURSE_ADD_LOCAL_USER
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: local-users
             {{- end }}
             {{- if .Values.concourse.web.tls.enabled }}
@@ -140,12 +141,12 @@ spec:
             - name: CONCOURSE_ENCRYPTION_KEY
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: encryption-key
             - name: CONCOURSE_OLD_ENCRYPTION_KEY
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: old-encryption-key
             {{- end }}
             {{- if .Values.concourse.web.debugBindIp }}
@@ -236,12 +237,12 @@ spec:
             - name: CONCOURSE_POSTGRES_USER
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: postgresql-user
             - name: CONCOURSE_POSTGRES_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: postgresql-password
             {{- if .Values.concourse.web.postgres.sslmode }}
             - name: CONCOURSE_POSTGRES_SSLMODE
@@ -288,18 +289,18 @@ spec:
             - name: CONCOURSE_AWS_SECRETSMANAGER_ACCESS_KEY
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: aws-secretsmanager-access-key
             - name: CONCOURSE_AWS_SECRETSMANAGER_SECRET_KEY
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: aws-secretsmanager-secret-key
             {{- if .Values.concourse.web.awsSecretsManager.keyAuth.useSessionToken }}
             - name: CONCOURSE_AWS_SECRETSMANAGER_SESSION_TOKEN
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: aws-secretsmanager-session-token
             {{- end }}
             {{- end }}
@@ -317,18 +318,18 @@ spec:
             - name: CONCOURSE_AWS_SSM_ACCESS_KEY
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: aws-ssm-access-key
             - name: CONCOURSE_AWS_SSM_SECRET_KEY
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: aws-ssm-secret-key
             {{- if .Values.concourse.web.awsSsm.keyAuth.useSessionToken }}
             - name: CONCOURSE_AWS_SSM_SESSION_TOKEN
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: aws-ssm-session-token
             {{- end }}
             {{- end }}
@@ -360,7 +361,7 @@ spec:
             - name: CONCOURSE_VAULT_CLIENT_TOKEN
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: vault-client-token
             {{- end }}
             {{- if eq .Values.concourse.web.vault.authBackend "cert" }}
@@ -373,7 +374,7 @@ spec:
             - name: CONCOURSE_VAULT_AUTH_PARAM
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: vault-client-auth-param
             {{- end }}
             {{- if .Values.concourse.web.vault.authBackendMaxTtl }}
@@ -461,7 +462,7 @@ spec:
             - name: CONCOURSE_INFLUXDB_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: influxdb-password
             - name: CONCOURSE_INFLUXDB_INSECURE_SKIP_VERIFY
               value: {{ .Values.concourse.web.influxdb.insecureSkipVerify | quote}}
@@ -622,12 +623,12 @@ spec:
             - name: CONCOURSE_CF_CLIENT_ID
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: cf-client-id
             - name: CONCOURSE_CF_CLIENT_SECRET
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: cf-client-secret
             {{- if .Values.concourse.web.auth.cf.apiUrl }}
             - name: CONCOURSE_CF_API_URL
@@ -646,12 +647,12 @@ spec:
             - name: CONCOURSE_GITHUB_CLIENT_ID
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: github-client-id
             - name: CONCOURSE_GITHUB_CLIENT_SECRET
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: github-client-secret
             {{- if .Values.concourse.web.auth.github.host }}
             - name: CONCOURSE_GITHUB_HOST
@@ -666,12 +667,12 @@ spec:
             - name: CONCOURSE_GITLAB_CLIENT_ID
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: gitlab-client-id
             - name: CONCOURSE_GITLAB_CLIENT_SECRET
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: gitlab-client-secret
             {{- if .Values.concourse.web.auth.gitlab.host }}
             - name: CONCOURSE_GITLAB_HOST
@@ -772,12 +773,12 @@ spec:
             - name: CONCOURSE_OAUTH_CLIENT_ID
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: oauth-client-id
             - name: CONCOURSE_OAUTH_CLIENT_SECRET
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: oauth-client-secret
             {{- if .Values.concourse.web.auth.oauth.authUrl }}
             - name: CONCOURSE_OAUTH_AUTH_URL
@@ -828,12 +829,12 @@ spec:
             - name: CONCOURSE_OIDC_CLIENT_ID
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: oidc-client-id
             - name: CONCOURSE_OIDC_CLIENT_SECRET
               valueFrom:
                 secretKeyRef:
-                  name: {{ template "concourse.concourse.fullname" . }}
+                  name: {{ template "concourse.web.fullname" . }}
                   key: oidc-client-secret
             {{- if .Values.concourse.web.auth.oidc.scope }}
             - name: CONCOURSE_OIDC_SCOPE
@@ -990,7 +991,7 @@ spec:
 {{- end }}
         - name: concourse-keys
           secret:
-            secretName: {{ template "concourse.concourse.fullname" . }}
+            secretName: {{ template "concourse.web.fullname" . }}
             defaultMode: 0400
             items:
               - key: host-key
@@ -1002,7 +1003,7 @@ spec:
         {{- if .Values.secrets.teamAuthorizedKeys }}
         - name: team-authorized-keys
           secret:
-            secretName: {{ template "concourse.concourse.fullname" . }}
+            secretName: {{ template "concourse.web.fullname" . }}
             defaultMode: 0400
             items:
             {{- range .Values.secrets.teamAuthorizedKeys }}
@@ -1013,7 +1014,7 @@ spec:
         {{- if .Values.concourse.web.tls.enabled }}
         - name: web-tls
           secret:
-            secretName: {{ template "concourse.concourse.fullname" . }}
+            secretName: {{ template "concourse.web.fullname" . }}
             defaultMode: 0400
             items:
               - key: web-tls-cert
@@ -1024,7 +1025,7 @@ spec:
         {{- if .Values.concourse.web.vault.enabled }}
         - name: vault-keys
           secret:
-            secretName: {{ template "concourse.concourse.fullname" . }}
+            secretName: {{ template "concourse.web.fullname" . }}
             defaultMode: 0400
             items:
             {{- if .Values.concourse.web.vault.useCaCert }}
@@ -1041,7 +1042,7 @@ spec:
         {{- if not (eq .Values.concourse.web.postgres.sslmode "disable") }}
         - name: postgresql-keys
           secret:
-            secretName: {{ template "concourse.concourse.fullname" . }}
+            secretName: {{ template "concourse.web.fullname" . }}
             defaultMode: 0400
             items:
               - key: postgresql-ca-cert
@@ -1054,15 +1055,15 @@ spec:
         {{- if .Values.concourse.web.syslog.enabled }}
         - name: syslog-keys
           secret:
-            secretName: {{ template "concourse.concourse.fullname" . }}
+            secretName: {{ template "concourse.web.fullname" . }}
             defaultMode: 0400
             items:
               - key: syslog-ca-cert
                 path: ca.cert
         {{- end }}
         - name: auth-keys
           secret:
-            secretName: {{ template "concourse.concourse.fullname" . }}
+            secretName: {{ template "concourse.web.fullname" . }}
             defaultMode: 0400
             items:
               {{- if .Values.concourse.web.auth.cf.useCaCert }}
@@ -1085,3 +1086,4 @@ spec:
               - key: oidc-ca-cert
                 path: oidc_ca.cert
               {{- end }}
+{{- end }}
diff --git a/stable/concourse/templates/web-ingress.yaml b/stable/concourse/templates/web-ingress.yaml
@@ -1,3 +1,4 @@
+{{- if .Values.web.enabled -}}
 {{- if .Values.web.ingress.enabled -}}
 {{- $releaseName := .Release.Name -}}
 {{- $serviceName := default "web" .Values.web.nameOverride -}}
@@ -30,3 +31,4 @@ spec:
 {{ toYaml .Values.web.ingress.tls | indent 4 }}
   {{- end -}}
 {{- end -}}
+{{- end -}}
diff --git a/stable/concourse/templates/web-role.yaml b/stable/concourse/templates/web-role.yaml
@@ -1,3 +1,4 @@
+{{- if .Values.web.enabled -}}
 {{- if .Values.rbac.create -}}
 {{- if .Values.concourse.web.kubernetes.enabled -}}
 apiVersion: rbac.authorization.k8s.io/{{ .Values.rbac.apiVersion }}
@@ -15,3 +16,4 @@ rules:
   verbs: ["get"]
 {{- end -}}
 {{- end -}}
+{{- end -}}
diff --git a/stable/concourse/templates/web-rolebinding.yaml b/stable/concourse/templates/web-rolebinding.yaml
@@ -1,3 +1,4 @@
+{{- if .Values.web.enabled -}}
 {{- if .Values.rbac.create -}}
 {{- if .Values.concourse.web.kubernetes.enabled -}}
 {{- range .Values.concourse.web.kubernetes.teams }}
@@ -23,3 +24,4 @@ subjects:
 {{- end }}
 {{- end -}}
 {{- end -}}
+{{- end -}}