Exec plugin overrides static certs in kubeconfig (different behavior than over statically configured credentials)

[ankeesler]

What happened:

• My exec plugin overrode the static certs I had in my kubeconfig.

What you expected to happen:

• The static certs I had in my kubeconfig should take precedent over whatever is configured by my exec plugin.

How to reproduce it (as minimally and precisely as possible):

With admin-like privileges, do the following.

# Generate CSR.
openssl req -nodes -subj /CN=some-username -newkey rsa:2048 -keyout /tmp/aaa.key -out /tmp/aaa.csr

# Create CSR.
cat <<EOF | kubectl replace -f -
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
  name: aaa
spec:
  request: $(cat /tmp/aaa.csr | base64 | tr -d '\n')
  signerName: kubernetes.io/kube-apiserver-client
  usages: [client auth]
EOF

# Issue certificate.
kubectl certificate approve aaa
kubectl get csr aaa -o jsonpath={.status.certificate} | base64 -d > /tmp/aaa.crt

# Make request with both certificate and exec plugin that returns token.
cat <<EOF >/tmp/aaa.sh
#!/bin/bash
echo '{"kind":"ExecCredential","apiVersion":"client.authentication.k8s.io/v1beta1","status":{"clientKeyData":"bad","clientCertificateData":"bad"}}'
EOF
chmod +x /tmp/aaa.sh
kubectl config set-credentials aaa --client-certificate=/tmp/aaa.crt --client-key=/tmp/aaa.key --exec-api-version client.authentication.k8s.io/v1beta1 --exec-command=/tmp/aaa.sh
kubectl --user aaa get pods

Anything else we need to know?:

• This seems like unexpected behavior because other statically configured credentials (i.e., basic auth, bearer token) take precedent over the stuff returned by the exec plugin.
• This was pointed out by @enj when he was reviewing some exec plugin integration tests.

Environment:

• Kubernetes version (use kubectl version): 1.20.0
• Cloud provider or hardware configuration: kind, but shouldn't matter
• OS (e.g: cat /etc/os-release): Darwin, but shouldn't matter
• Kernel (e.g. uname -a): Darwin, but shouldn't matter
• Install tools:
• Network plugin and version (if this is a network-related bug):
• Others:

[ankeesler]

/sig auth

[enj]

This should also cover the direct case right?

kubectl get ns --client-...=

$ kubectl options |& grep TLS
      --client-certificate='': Path to a client certificate file for TLS
      --client-key='': Path to a client key file for TLS

Precedence should be:

1. CLI flags
2. Static config
3. Exec plugin

[enj]

/triage accepted

[enj]

/priority important-soon

[margocrawf]

/assign

[liggitt]

is this required for exec auth GA in 1.22?

[ankeesler]

is this required for exec auth GA in 1.22?

I asked @enj that same question week before last (I've been on PTO). @enj posited that this is a "nice to have" (i.e., it shouldn't block GA) since there are other CLI cred override weirdisms.

[enj]

is this required for exec auth GA in 1.22?

I asked @enj that same question week before last (I've been on PTO). @enj posited that this is a "nice to have" (i.e., it shouldn't block GA) since there are other CLI cred override weirdisms.

Based on what I observed in kubernetes-sigs/kind#2229 (comment), it looks like --token does not override client certs in static config, which is wrong. In general we need to fix credential precedence handling in kubectl per #99603 (comment).

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[enj]

/remove-lifecycle rotten
/lifecycle frozen