New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"the server has asked for the client to provide credentials" #277

Closed
jaxxstorm opened this Issue Jan 8, 2018 · 8 comments

Comments

Projects
None yet
4 participants
@jaxxstorm

jaxxstorm commented Jan 8, 2018

What happened:

When using a kubeconfig that contains many clusters, contexts and users, if the current context does not match the current env, ksonnet returns ERROR the server has asked for the client to provide credentials

What you expected to happen:

Applying ksonnet envs shouldn't rely on the context being set with kubectl

How to reproduce it (as minimally and precisely as possible):

  • Add two envs, X and Y
  • Set the current context to Y
  • ks apply x

Anything else we need to know?:

This may be just a misunderstanding on my part, but if ks has the ability to know about servers, namespaces and other things, when creating an env, it should have enough information to apply the changes. Perhaps if ks stored the user as well, it would be more useful.

The reason I think this is important is because in a CI pipeline, it would be useful to not have to specify kubectl set-context every time for each environment.

Environment:

  • ksonnet version (use ks version): ksonnet version: v0.8.0
  • Kubernetes version (use kubectl version): Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.4", GitCommit:"9befc2b8928a9426501d3bf62f72849d5cbcd5a3", GitTreeState:"clean", BuildDate:"2017-11-20T05:28:34Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"darwin/amd64"} Server Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.4", GitCommit:"9befc2b8928a9426501d3bf62f72849d5cbcd5a3", GitTreeState:"clean", BuildDate:"2017-11-20T05:17:43Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
@hausdorff

This comment has been minimized.

Contributor

hausdorff commented Jan 8, 2018

@jaxxstorm Thanks for the report, that should not be the case! I can't seem to reproduce it, but I suspect that it's trying to authenticate with incorrect user credentials, so: are you using different users in X and Y? If not, could you share X and Y (without the key information, of course :) )?

@jaxxstorm

This comment has been minimized.

jaxxstorm commented Jan 8, 2018

Here's my kubeconfig:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: REDACTED
    server: https://<ip>
  name: lbrlabs
- cluster:
    certificate-authority-data: REDACTED
    server: https://<ip>
  name: tukd1
- cluster:
    certificate-authority-data: REDACTED
    server: https://<ip>
  name: uw2d1
- cluster:
    certificate-authority-data: REDACTED
    server: https://<ip>
  name: uw2d2
contexts:
- context:
    cluster: lbrlabs
    user: lbrlabs
  name: lbrlabs
- context:
    cluster: tukd1
    user: tukd1
  name: tukd1
- context:
    cluster: uw2d1
    namespace: default
    user: uw2d1-admin
  name: uw2d1
- context:
    cluster: uw2d2
    namespace: ks-dev
    user: uw2d2-admin
  name: uw2d2
current-context: uw2d1
kind: Config
preferences: {}
users:
- name: lbrlabs
  user:
    auth-provider:
      config:
        access-token: REDACTED
        cmd-args: config config-helper --format=json
        cmd-path: /Users/Lee/Downloads/google-cloud-sdk/bin/gcloud
        expiry: 2018-01-03T04:34:19Z
        expiry-key: '{.credential.token_expiry}'
        token-key: '{.credential.access_token}'
      name: gcp
- name: tukd1
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED
- name: uw2d1-admin
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED
- name: uw2d2-admin
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED

In this case, X is uw2d1 and Y is uw2d2. If there's anything else I can provide, let me know

@abiogenesis-now

This comment has been minimized.

Contributor

abiogenesis-now commented Jan 11, 2018

@hausdorff can confirm that I have encountered this issue as well (e.g. when using a kubeconfig file that references 2 clusters)

@jaxxstorm jaxxstorm changed the title from "the server has asked for the client to provide credentials to "the server has asked for the client to provide credentials" Jan 11, 2018

@hausdorff

This comment has been minimized.

Contributor

hausdorff commented Jan 11, 2018

@abiogenesis-now Ok, I'll look after I finish reviewing PRs.

@hausdorff hausdorff modified the milestone: 0.9.0 Jan 11, 2018

@hausdorff

This comment has been minimized.

Contributor

hausdorff commented Jan 17, 2018

@jaxxstorm Ok, so what's happening is that ks doesn't know which user to apply with, so it will just choose whatever the active user is in your relevant kubeconfig file. This is tricky because environment specifications are meant to be checked in, which means user-specific stuff can't be in there.

I believe this is a fairly common case, so we will need to design some kind of a workflow for how to bind users to different environments.

@jessicayuen @bryanl, thoughts?

@jaxxstorm

This comment has been minimized.

jaxxstorm commented May 30, 2018

@hausdorff did we ever get this looked at?

@bryanl

This comment has been minimized.

Member

bryanl commented May 30, 2018

@jaxxstorm I'd like to review what we have here. If I understand correctly, it looks like you are having issues with ksonnet determining the correct context to run with. As it stands right now, we store the cluster's API server URL and a namespace. This is not optimal. In a better world, we should expose multiple ways of identifying a cluster (including specifying a context). This task isn't trivial, but should be straight forward.

In the mean time, does specifying the --context flag work? I could see this working in every case except for diff since there are cases where you could be using two contexts.

The next steps are to:

  • figure out a better way to describe clusters in app.yaml
  • account for multiple contexts in git
  • possibly think of other ways to override contexts
@bryanl

This comment has been minimized.

Member

bryanl commented Sep 10, 2018

@jaxxstorm I'm closing this issue due to age. Please reopen if needed.

@bryanl bryanl closed this Sep 10, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment