Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to specify cluster by cluster name rather than IP #58

Closed
rfratto opened this issue Aug 18, 2019 · 6 comments

Comments

@rfratto
Copy link
Member

commented Aug 18, 2019

For testing, I sometimes launch a temporary local single-node Kubernetes cluster which receives a different IP address every time it is created. This makes deploying to it with tanka a bit difficult as the apiServer field in spec.json requires a hostname.

I'd like to be able to optionally specify clusters by name rather than the IP address.

For example, given the ~/.kube/config file of:

apiVersion: v1
clusters:
- cluster:
    certificate-authority: /Users/robert/.minikube/ca.crt
    server: https://192.168.64.3:8443
  name: minikube
# ... 

I would like to be able to refer to that API server in the spec.json in one of these ways:

{
  "apiVersion": "tanka.dev/v1alpha1",
  "kind": "Environment",
  "spec": {
    // option 1: use cluster name in apiServer 
    "apiServer": "minikube",
  
    // option 2: a separate key that is used if apiServer is empty 
    "clusterName": "minikube",
  
    "namespace": "default"
  }
}

WDYT @sh0rez?

@issue-label-bot

This comment has been minimized.

Copy link

commented Aug 18, 2019

Issue-Label Bot is automatically applying the label kind/feature to this issue, with a confidence of 0.98. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

@malcolmholmes

This comment has been minimized.

Copy link
Collaborator

commented Aug 18, 2019

@rfratto the apiServer param will be a URL, therefore you can specify http://minikube just as well as specifying an IP Address.

@rfratto

This comment has been minimized.

Copy link
Member Author

commented Aug 18, 2019

@malcolmholmes right, I guess that would fix the problem with tanka but I feel it just shifts the issue elsewhere: in the general case, on each machine I'd still have to manage the hostname resolution (in /etc/resolv.conf?) every time the IP updates.

That's fine, but I still think what I'm asking for here may be useful and easier to work with since tools like minikube are already updating ~/.kube/config with the latest IP address.

It looks like something along these lines has been requested for ksonnet as well.

rfratto added a commit to rfratto/tanka that referenced this issue Aug 18, 2019
@malcolmholmes

This comment has been minimized.

Copy link
Collaborator

commented Aug 18, 2019

So you're asking to use a kubeconfig cluster in spec.json instead of the URL?

The issue with this is that kubeconfig is user configurable - it can differ across users, and is therefore not guaranteed consistent. This could lead to code being incorrectly deployed to the wrong cluster if a user incorrectly configured their KUBECONFIG.

@rfratto

This comment has been minimized.

Copy link
Member Author

commented Aug 18, 2019

Yeah, the fact that it is user configurable and can differ across users is actually the behavior I want - aside from the IP changing whenever I recreate the cluster, I'm also trying to re-use the same ksonnet on multiple machines running minikube, where the only thing that's identical between them is the cluster name in KUBECONFIG.

I view this as a secondary behavior for local development rather than the preferred way to specify environments. Feel free to close this if this just isn't something that should be supported, though; I can make tooling to handle this for my specific use case.

@sh0rez

This comment has been minimized.

Copy link
Member

commented Aug 25, 2019

I see the point of having a generic way to specify a cluster by name, but I value @malcolmholmes concern higher: Because it makes the cluster ambiguous, it works against our efforts to prevent applying the correct config to the wrong cluster. Only an IP really uniquely identifies a cluster.

IMHO it is the job of a correct DNS setup to deal with differences across environments (that is /etc/resolv.conf, /etc/hosts or some other way of local DNS.

I personally like using k3d for my local Kubernetes needs (https://github.com/rancher/k3d), because it allows to run the whole cluster inside of a docker container and port-binds the apiServer to localhost:6443.

@sh0rez sh0rez closed this Aug 25, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.