-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
First tutorial page: getting started with GCE #3528
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,188 @@ | ||
# Getting Started with kops on GCE | ||
|
||
Make sure you have [installed kops](../install.md) and [installed kubectl](../install.md), and installed | ||
the [gcloud tools](https://cloud.google.com/sdk/downloads). | ||
|
||
You'll need a Google Cloud account, and make sure that gcloud is logged in to your account using `gcloud init`. | ||
|
||
You should confirm that basic commands like `gcloud compute zones list` are working. | ||
|
||
You'll also need to [configure default credentials](https://developers.google.com/accounts/docs/application-default-credentials), using `gcloud auth application-default login`. | ||
|
||
<!-- TODO: Can we get rid of `gcloud auth application-default login` ? --> | ||
|
||
# Creating a state store | ||
|
||
kops needs a state store, to hold the configuration for your clusters. The simplest configuration | ||
for Google Cloud is to store it in a Google Cloud Storage bucket in the same account, so that's how we'll | ||
start. | ||
|
||
So, just create an empty bucket - you can use any name: `gsutil mb s3://kubernetes-clusters/` | ||
|
||
# Creating our first cluster | ||
|
||
`kops create cluster` creates the Cluster and InstanceGroup objects you'll be working with in kops: | ||
|
||
``` | ||
PROJECT=`gcloud config get-value project` | ||
export KOPS_FEATURE_FLAGS=AlphaAllowGCE # to unlock the GCE features | ||
kops create cluster simple.k8s.local --zones us-central1-a --state gs://kubernetes-clusters/ --project=${PROJECT} | ||
``` | ||
|
||
You can now list the clusters in your kops state store (the GCS bucket we created): | ||
|
||
`kops get cluster --state gs://kubernetes-clusters/` | ||
|
||
``` | ||
NAME CLOUD ZONES | ||
simple.k8s.local gce us-central1-a | ||
``` | ||
|
||
<!-- TODO: Fix bug where zones not showing up --> | ||
|
||
This shows that you have one Cluster configured, named `simple.k8s.local`. The Cluster holds the cluster-wide configuration for | ||
a kubernetes cluster - things like the kubernetes version, and the authorization policy in use. | ||
|
||
The `kops` tool should feel a lot like `kubectl` - kops uses the same API machinery as kubernetes, | ||
so it should behave similarly, although now you are managing kubernetes clusters, instead of managing | ||
objects on a kubernetes cluster. | ||
|
||
You can see the details of your Cluster by doing: | ||
|
||
`> kops get cluster --state gs://kubernetes-clusters/ simple.k8s.local -oyaml` | ||
``` | ||
apiVersion: kops/v1alpha2 | ||
kind: Cluster | ||
metadata: | ||
creationTimestamp: 2017-10-03T05:07:27Z | ||
name: simple.k8s.local | ||
spec: | ||
api: | ||
loadBalancer: | ||
type: Public | ||
authorization: | ||
alwaysAllow: {} | ||
channel: stable | ||
cloudProvider: gce | ||
configBase: gs://kubernetes-clusters/simple.k8s.local | ||
etcdClusters: | ||
- etcdMembers: | ||
- instanceGroup: master-us-central1-a | ||
name: a | ||
name: main | ||
- etcdMembers: | ||
- instanceGroup: master-us-central1-a | ||
name: a | ||
name: events | ||
iam: | ||
legacy: false | ||
kubernetesApiAccess: | ||
- 0.0.0.0/0 | ||
kubernetesVersion: 1.7.2 | ||
masterPublicName: api.simple.k8s.local | ||
networking: | ||
kubenet: {} | ||
nonMasqueradeCIDR: 100.64.0.0/10 | ||
project: my-gce-project | ||
sshAccess: | ||
- 0.0.0.0/0 | ||
subnets: | ||
- name: us-central1 | ||
region: us-central1 | ||
type: Public | ||
topology: | ||
dns: | ||
type: Public | ||
masters: public | ||
nodes: public | ||
``` | ||
|
||
Similarly, you can also see your InstanceGroups using: | ||
|
||
`kops get instancegroup --state gs://kubernetes-clusters/ --name simple.k8s.local` | ||
``` | ||
NAME ROLE MACHINETYPE MIN MAX SUBNETS | ||
master-us-central1-a Master n1-standard-1 1 1 us-central1 | ||
nodes Node n1-standard-2 2 2 us-central1 | ||
``` | ||
|
||
<!-- TODO: Fix subnets vs regions --> | ||
|
||
InstanceGroups are the other main kops object - an InstanceGroup manages a set of cloud instances, | ||
which then are registered in kubernetes as Nodes. You have multiple InstanceGroups for different types | ||
of instances / Nodes - in our simple example we have one for our master (which only has a single member), | ||
and one for our nodes (and we have two nodes configured). | ||
|
||
We'll see a lot more of Clusters and InstanceGroups as we use kops to reconfigure clusters. But let's get | ||
on with our first cluster. | ||
|
||
# Export KOPS_STATE_STORE | ||
|
||
Rather than typing the `--state` argument every time, it's much easier to export the `KOPS_STATE_STORE` | ||
environment variable: | ||
|
||
``` | ||
export KOPS_STATE_STORE=gs://kubernetes-clusters/ | ||
``` | ||
|
||
You can also put this in your `~/.bashrc` or similar. | ||
|
||
# Creating a cluster | ||
|
||
`kops create cluster` created the Cluster & InstanceGroup objects in our state store, | ||
but didn't actually create any instances or other cloud objects in GCE. To do that, we'll use | ||
`kops update cluster`. | ||
|
||
`kops update cluster` without `--yes` will show you a preview of all the changes will be made; | ||
it is very useful to see what kops is about to do, before actually making the changes. | ||
|
||
Run `kops update cluster simple.k8s.local` and peruse the changes. | ||
|
||
We're now finally ready to create the object: `kops update cluster simple.k8s.local --yes` | ||
|
||
(If you haven't created an SSH key, you'll have to `ssh-keygen -t rsa`) | ||
|
||
<!-- TODO: We don't need this on GCE; remove SSH key requirement --> | ||
|
||
Your cluster is created in the background - kops actually creates GCE Managed Instance Groups | ||
that run the instances; this ensures that even if instances are terminated, they will automatically | ||
be relaunched by GCE and your cluster will self-heal. | ||
|
||
After a few minutes, you should be able to do `kubectl get nodes` and your first cluster should be ready! | ||
|
||
# Enjoy | ||
|
||
At this point you have a kubernetes cluster - the core commands to do so are as simple as `kops create cluster` | ||
and `kops update cluster`. There's a lot more power in kops, and even more power in kubernetes itself, so we've | ||
put a few jumping off places here. But when you're done, don't forget to [delete your cluster](#deleting-the-cluster). | ||
|
||
* [Manipulate InstanceGroups](working-with-instancegroups.md) to add more nodes, change image | ||
|
||
# Deleting the cluster | ||
|
||
When you're done using the cluster, you should delete it to release the cloud resources. `kops delete cluster` is | ||
the command. When run without `--yes` it shows a preview of the objects it will delete: | ||
|
||
``` | ||
> kops delete cluster simple.k8s.local | ||
TYPE NAME ID | ||
Address api-simple-k8s-local api-simple-k8s-local | ||
Disk a-etcd-events-simple-k8s-local a-etcd-events-simple-k8s-local | ||
Disk a-etcd-main-simple-k8s-local a-etcd-main-simple-k8s-local | ||
ForwardingRule api-simple-k8s-local api-simple-k8s-local | ||
Instance master-us-central1-a-9847 us-central1-a/master-us-central1-a-9847 | ||
Instance nodes-0s0w us-central1-a/nodes-0s0w | ||
Instance nodes-dvlq us-central1-a/nodes-dvlq | ||
InstanceGroupManager a-master-us-central1-a-simple-k8s-local us-central1-a/a-master-us-central1-a-simple-k8s-local | ||
InstanceGroupManager a-nodes-simple-k8s-local us-central1-a/a-nodes-simple-k8s-local | ||
InstanceTemplate master-us-central1-a-simple-k8s-local-1507008700 master-us-central1-a-simple-k8s-local-1507008700 | ||
InstanceTemplate nodes-simple-k8s-local-1507008700 nodes-simple-k8s-local-1507008700 | ||
Route simple-k8s-local-715bb0c7-a7fc-11e7-93d7-42010a800002 simple-k8s-local-715bb0c7-a7fc-11e7-93d7-42010a800002 | ||
Route simple-k8s-local-9a2a08e8-a7fc-11e7-93d7-42010a800002 simple-k8s-local-9a2a08e8-a7fc-11e7-93d7-42010a800002 | ||
Route simple-k8s-local-9c17a4e6-a7fc-11e7-93d7-42010a800002 simple-k8s-local-9c17a4e6-a7fc-11e7-93d7-42010a800002 | ||
TargetPool api-simple-k8s-local api-simple-k8s-local | ||
|
||
Must specify --yes to delete cluster | ||
``` | ||
|
||
After you've double-checked you're deleting exactly what you want to delete, run `kops delete cluster simple.k8s.local --yes`. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,128 @@ | ||
# Upgrading kubernetes | ||
|
||
Upgrading kubernetes is very easy with kops, as long as you are using a compatible version of kops. | ||
The kops `1.8.x` series (for example) supports the kubernetes 1.6, 1.7 and 1.8 series, | ||
as per the kubernetes deprecation policy. Older versions of kubernetes will likely still work, but these | ||
are on a best-effort basis and will have little if any testing. kops `1.8` will not support the kubernetes | ||
`1.9` series, and for full support of kubernetes `1.9` it is best to wait for the kops `1.9` series release. | ||
We aim to release the next major version of kops within a few weeks of the equivalent major release of kubernetes, | ||
so kops `1.9.0` will be released within a few weeks of kubernetes `1.9.0`. We try to ensure that a 1.9 pre-release | ||
(alpha or beta) is available at the kubernetes release, for early adopters. | ||
|
||
Upgrading kubernetes is similar to changing the image on an InstanceGroup, except that the kubernetes version is | ||
controlled at the cluster level. So instead of `kops edit ig <name>`, we `kops edit cluster`, and change the | ||
`kubernetesVersion` field. `kops edit cluster` will open your editor with the cluster, similar to: | ||
|
||
``` | ||
# Please edit the object below. Lines beginning with a '#' will be ignored, | ||
# and an empty file will abort the edit. If an error occurs while saving this file will be | ||
# reopened with the relevant failures. | ||
# | ||
apiVersion: kops/v1alpha2 | ||
kind: Cluster | ||
metadata: | ||
creationTimestamp: 2017-10-04T03:52:25Z | ||
name: simple.k8s.local | ||
spec: | ||
api: | ||
loadBalancer: | ||
type: Public | ||
authorization: | ||
alwaysAllow: {} | ||
channel: stable | ||
cloudProvider: gce | ||
configBase: gs://kubernetes-clusters/simple.k8s.local | ||
etcdClusters: | ||
- etcdMembers: | ||
- instanceGroup: master-us-central1-a | ||
name: a | ||
name: main | ||
- etcdMembers: | ||
- instanceGroup: master-us-central1-a | ||
name: a | ||
name: events | ||
iam: | ||
legacy: false | ||
kubernetesApiAccess: | ||
- 0.0.0.0/0 | ||
kubernetesVersion: 1.7.2 | ||
masterInternalName: api.internal.simple.k8s.local | ||
masterPublicName: api.simple.k8s.local | ||
networking: | ||
kubenet: {} | ||
nonMasqueradeCIDR: 100.64.0.0/10 | ||
project: gce-project | ||
sshAccess: | ||
- 0.0.0.0/0 | ||
subnets: | ||
- name: us-central1 | ||
region: us-central1 | ||
type: Public | ||
topology: | ||
dns: | ||
type: Public | ||
masters: public | ||
nodes: public | ||
``` | ||
|
||
Edit `kubernetesVersion`, changing it to `1.7.7` for example. | ||
|
||
|
||
Apply the changes to the cloud infrastructure using `kops update cluster` and `kops update cluster --yes`: | ||
|
||
``` | ||
Will create resources: | ||
InstanceTemplate/master-us-central1-a-simple-k8s-local | ||
Network name:default id:default | ||
Tags [simple-k8s-local-k8s-io-role-master] | ||
Preemptible false | ||
BootDiskImage cos-cloud/cos-stable-57-9202-64-0 | ||
BootDiskSizeGB 64 | ||
BootDiskType pd-standard | ||
CanIPForward true | ||
Scopes [compute-rw, monitoring, logging-write, storage-ro, https://www.googleapis.com/auth/ndev.clouddns.readwrite] | ||
Metadata {cluster-name: <resource>, startup-script: <resource>} | ||
MachineType n1-standard-1 | ||
|
||
InstanceTemplate/nodes-simple-k8s-local | ||
Network name:default id:default | ||
Tags [simple-k8s-local-k8s-io-role-node] | ||
Preemptible false | ||
BootDiskImage debian-cloud/debian-9-stretch-v20170918 | ||
BootDiskSizeGB 128 | ||
BootDiskType pd-standard | ||
CanIPForward true | ||
Scopes [compute-rw, monitoring, logging-write, storage-ro] | ||
Metadata {startup-script: <resource>, cluster-name: <resource>} | ||
MachineType n1-standard-2 | ||
|
||
Will modify resources: | ||
InstanceGroupManager/us-central1-a-master-us-central1-a-simple-k8s-local | ||
InstanceTemplate id:master-us-central1-a-simple-k8s-local-1507089163 -> name:master-us-central1-a-simple-k8s-local | ||
|
||
InstanceGroupManager/us-central1-a-nodes-simple-k8s-local | ||
InstanceTemplate id:nodes-simple-k8s-local-1507089694 -> name:nodes-simple-k8s-local | ||
``` | ||
|
||
|
||
`kops rolling-update cluster` will show that all nodes need to be restarted. | ||
|
||
``` | ||
NAME STATUS NEEDUPDATE READY MIN MAX NODES | ||
master-us-central1-a NeedsUpdate 1 0 1 1 1 | ||
nodes NeedsUpdate 3 0 3 3 3 | ||
``` | ||
|
||
Restart the instances with `kops rolling-update cluster --yes`. | ||
|
||
``` | ||
> kubectl get nodes -owide | ||
NAME STATUS AGE VERSION EXTERNAL-IP OS-IMAGE KERNEL-VERSION | ||
master-us-central1-a-8fcc Ready 26m v1.7.7 35.194.56.129 Container-Optimized OS from Google 4.4.35+ | ||
nodes-9cml Ready 16m v1.7.7 35.193.12.73 Ubuntu 16.04.3 LTS 4.10.0-35-generic | ||
nodes-km98 Ready 10m v1.7.7 35.194.25.144 Ubuntu 16.04.3 LTS 4.10.0-35-generic | ||
nodes-wbb2 Ready 2m v1.7.7 35.188.177.16 Ubuntu 16.04.3 LTS 4.10.0-35-generic | ||
``` | ||
|
||
<!-- TODO: Do we drain, validate and then restart --> | ||
<!-- TODO: Fix timings in rolling update --> |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not that I have found with the API