Add support for deploying without using Kubernetes secrets

[marcleblanc2]

...and to disable RBAC resources.

Customer's Kubernetes security policies block the creation of secrets and RBAC resources

This PR doesn't change any default behaviour which would impact other customers, only adds a couple new configs customer can choose to use.

Checklist

• Follow the manual testing process
• Update changelog
• Update Kubernetes update doc

Test plan

Tested with customer

[jdpleiness]

Looks good overall. I think there is one minor issue around validation though that I commented on 🚀

[jdpleiness]

This looks fine, but there's a case where someone disables secrets and forgets to set this, it's just going to return empty vars and fail at runtime.
We might want to add some validation here, possible something like:

{{- define "sourcegraph.redisConnection" -}}
{{- if .Values.sourcegraph.disableKubernetesSecrets -}}
- name: REDIS_CACHE_ENDPOINT
  value: {{ required "sourcegraph.redisCacheEndpoint is required when sourcegraph.disableKubernetesSecrets is true" .Values.sourcegraph.redisCacheEndpoint }}
- name: REDIS_STORE_ENDPOINT
  value: {{ required "sourcegraph.redisStoreEndpoint is required when sourcegraph.disableKubernetesSecrets is true" .Values.sourcegraph.redisStoreEndpoint }}
{{- else -}}
- name: REDIS_CACHE_ENDPOINT
  valueFrom:
    secretKeyRef:
      key: endpoint
      name: {{ default .Values.redisCache.name .Values.redisCache.connection.existingSecret }}
- name: REDIS_STORE_ENDPOINT
  valueFrom:
    secretKeyRef:
      key: endpoint
      name: {{ default .Values.redisStore.name .Values.redisStore.connection.existingSecret }}
{{- end -}}
{{- end -}}

[marcleblanc2]

Great call out, I had considered how to approach this... if they're externalizing Redis, then this endpoint value would likely include credentials, which they'd want to define as REDIS_CACHE_ENDPOINT and REDIS_STORE_ENDPOINT env vars on the needed pods, using the same method that they're using to side-load the Postgres credentials as env vars. So we end up with 3 separate config methods for Redis endpoints, the same as Postgres credentials:

1. Kubernetes secrets (default)

2. sourcegraph.redisCacheEndpoint / sourcegraph.redisStoreEndpoint values in the override file

3. Inject the REDIS_CACHE_ENDPOINT and REDIS_STORE_ENDPOINT env vars on the needed pods

In this case, with the override file shared in Slack, this customer is using our Redis pods, so option 2 works for them, but if they decide to externalize Redis, then they'd have to switch to option 3.

[marcleblanc2]

Found a cleaner way to do this, re-using existing configs in the Helm chart:

.Values.redisCache.connection.endpoint
.Values.redisStore.connection.endpoint

When the customer sets .Values.sourcegraph.disableKubernetesSecrets: true, then these same, existing configs, already with the correct defaults, get fed in directly as env vars, instead of first getting fed into the creation of secret objects, just to be read back in as env vars from the secrets.

Then, if they want to define these env vars externally, ex. external redis with creds in the endpoint string, then they can set the two endpoint values to "".