-
Notifications
You must be signed in to change notification settings - Fork 17
Description
Unless one explicitly sets user in k8s_deploy(), it is assumed a username matching the cluster can be found:
rules_gitops/skylib/kustomize/kustomize.bzl
Line 509 in 39f94ee
| user_arg = """$(kubectl config view -o jsonpath='{.users[?(@.name == '"\\"${CLUSTER}\\")].name}")""" |
(My jsonpath-foo isn't strong, but I read this as "return the .name of the user who's .name == $CLUSTER - I feel I'm missing something obvious here?)
On our setup this fails as we sometimes have multiple users per cluster. We don't like running with admin-powers on by default, so everyone has <username>-<clustername>, and some has admin-<clustername> as backups. The query above returns an empty string on our setups, so things only work when the user happens to be in the right kubernetes context, so the blank user-name doesn't make a difference:
kubectl config use-context $USER-$OTHER_CLUSTER
bazel run :deploy-to-kubernetes
...
error: error validating "STDIN": error validating data: failed to download openapi: the server has asked for the client to provide credentials; if you choose to ignore these errors, turn validation off with --validate=false
kubectl config use-context $USER-$CORRECT_CLUSTER
bazel run :deploy-to-kubernetes
...
service/xxxx unchanged
deployment.apps/xxxx configured
sealedsecret.bitnami.com/xxxx unchanged
ingress.networking.k8s.io/xxxx unchangedKubernetes config does have contexts tying users, clusters and - optionally - namespaces together (which is also what rules_k8s relies on):
k8s_deploy(
# ...
context = "morten-siebuhr-some-cluster-name"
)Which is then passed to kubectl --context $CONTEXT?
Docs: https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/