Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update docker-multinode instructions and version #18894

Merged
merged 1 commit into from
Jan 12, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions cluster/get-kube-local.sh
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,8 @@ function create_cluster {
--api-servers=http://localhost:8080 \
--config=/etc/kubernetes/manifests \
--allow-privileged=true \
--cluster-dns=10.0.0.10 \
--cluster-domain=cluster.local \
--v=2"

echo -e -n "\tWaiting for master components to start..."
Expand Down
2 changes: 2 additions & 0 deletions cluster/images/hyperkube/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ COPY hyperkube /hyperkube
RUN chmod a+rx /hyperkube

COPY master-multi.json /etc/kubernetes/manifests-multi/master.json
COPY kube-proxy.json /etc/kubernetes/manifests-multi/kube-proxy.json

COPY master.json /etc/kubernetes/manifests/master.json
COPY etcd.json /etc/kubernetes/manifests/etcd.json
COPY kube-proxy.json /etc/kubernetes/manifests/kube-proxy.json
Expand Down
20 changes: 10 additions & 10 deletions docs/getting-started-guides/docker-multinode.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,6 @@ _Note_:
These instructions are somewhat significantly more advanced than the [single node](docker.md) instructions. If you are
interested in just starting to explore Kubernetes, we recommend that you start there.

_Note_:
There is a [bug](https://github.com/docker/docker/issues/14106) in Docker 1.7.0 that prevents this from working correctly.
Please install Docker 1.6.2 or Docker 1.7.1.

**Table of Contents**

- [Prerequisites](#prerequisites)
Expand All @@ -53,7 +49,7 @@ Please install Docker 1.6.2 or Docker 1.7.1.

## Prerequisites

1. You need a machine with docker of right version installed.
The only thing you need is a machine with **Docker 1.7.1 or higher**

## Overview

Expand All @@ -74,10 +70,13 @@ This pattern is necessary because the `flannel` daemon is responsible for settin
all of the Docker containers created by Kubernetes. To achieve this, it must run outside of the _main_ Docker daemon. However,
it is still useful to use containers for deployment and management, so we create a simpler _bootstrap_ daemon to achieve this.

You can specify k8s version on very node before install:
You can specify the version on every node before install:

```
export K8S_VERSION=<your_k8s_version (e.g. 1.0.3)>
```sh
export K8S_VERSION=<your_k8s_version (e.g. 1.1.3)>
export ETCD_VERSION=<your_etcd_version (e.g. 2.2.1)>
export FLANNEL_VERSION=<your_flannel_version (e.g. 0.5.5)>
export FLANNEL_IFACE=<flannel_interface (defaults to eth0)>
```

Otherwise, we'll use latest `hyperkube` image as default k8s version.
Expand All @@ -86,7 +85,8 @@ Otherwise, we'll use latest `hyperkube` image as default k8s version.

The first step in the process is to initialize the master node.

Clone the Kubernetes repo, and run [master.sh](docker-multinode/master.sh) on the master machine with root:
The MASTER_IP step here is optional, it defaults to the first value of `hostname -I`.
Clone the Kubernetes repo, and run [master.sh](docker-multinode/master.sh) on the master machine _with root_:

```console
$ export MASTER_IP=<your_master_ip (e.g. 1.2.3.4)>
Expand All @@ -102,7 +102,7 @@ See [here](docker-multinode/master.md) for detailed instructions explanation.

Once your master is up and running you can add one or more workers on different machines.

Clone the Kubernetes repo, and run [worker.sh](docker-multinode/worker.sh) on the worker machine with root:
Clone the Kubernetes repo, and run [worker.sh](docker-multinode/worker.sh) on the worker machine _with root_:

```console
$ export MASTER_IP=<your_master_ip (e.g. 1.2.3.4)>
Expand Down
63 changes: 46 additions & 17 deletions docs/getting-started-guides/docker-multinode/master.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,17 @@ Documentation for other releases can be found at

We'll begin by setting up the master node. For the purposes of illustration, we'll assume that the IP of this machine
is `${MASTER_IP}`. We'll need to run several versioned Kubernetes components, so we'll assume that the version we want
to run is `${K8S_VERSION}`, which should hold a value such as "1.0.7".
to run is `${K8S_VERSION}`, which should hold a value such as "1.1.3".

Enviroinment variables used:

```sh
export MASTER_IP=<the_master_ip_here>
export K8S_VERSION=<your_k8s_version (e.g. 1.1.3)>
export ETCD_VERSION=<your_etcd_version (e.g. 2.2.1)>
export FLANNEL_VERSION=<your_flannel_version (e.g. 0.5.5)>
export FLANNEL_IFACE=<flannel_interface (defaults to eth0)>
```

There are two main phases to installing the master:
* [Setting up `flanneld` and `etcd`](#setting-up-flanneld-and-etcd)
Expand All @@ -45,10 +55,9 @@ There are two main phases to installing the master:
## Setting up flanneld and etcd

_Note_:
There is a [bug](https://github.com/docker/docker/issues/14106) in Docker 1.7.0 that prevents this from working correctly.
Please install Docker 1.6.2 or Docker 1.7.1 or Docker 1.8.3.
This guide expects **Docker 1.7.1 or higher**.

### Setup Docker-Bootstrap
### Setup Docker Bootstrap

We're going to use `flannel` to set up networking between Docker daemons. Flannel itself (and etcd on which it relies) will run inside of
Docker containers themselves. To achieve this, we need a separate "bootstrap" instance of the Docker daemon. This daemon will be started with
Expand All @@ -70,13 +79,22 @@ across reboots and failures.
Run:

```sh
sudo docker -H unix:///var/run/docker-bootstrap.sock run --net=host -d gcr.io/google_containers/etcd:2.2.1 /usr/local/bin/etcd --listen-client-urls=http://127.0.0.1:4001,http://${MASTER_IP}:4001 --advertise-client-urls=http://${MASTER_IP}:4001 --data-dir=/var/etcd/data
sudo docker -H unix:///var/run/docker-bootstrap.sock run -d \
--net=host \
gcr.io/google_containers/etcd:${ETCD_VERSION} \
/usr/local/bin/etcd \
--listen-client-urls=http://127.0.0.1:4001,http://${MASTER_IP}:4001 \
--advertise-client-urls=http://${MASTER_IP}:4001 \
--data-dir=/var/etcd/data
```

Next, you need to set a CIDR range for flannel. This CIDR should be chosen to be non-overlapping with any existing network you are using:

```sh
sudo docker -H unix:///var/run/docker-bootstrap.sock run --net=host gcr.io/google_containers/etcd:2.2.1 etcdctl set /coreos.com/network/config '{ "Network": "10.1.0.0/16" }'
sudo docker -H unix:///var/run/docker-bootstrap.sock run \
--net=host \
gcr.io/google_containers/etcd:${ETCD_VERSION} \
etcdctl set /coreos.com/network/config '{ "Network": "10.1.0.0/16" }'
```


Expand Down Expand Up @@ -115,10 +133,16 @@ or it may be something else.
Now run flanneld itself:

```sh
sudo docker -H unix:///var/run/docker-bootstrap.sock run -d --net=host --privileged -v /dev/net:/dev/net quay.io/coreos/flannel:0.5.5 --ip-masq
sudo docker -H unix:///var/run/docker-bootstrap.sock run -d \
--net=host \
--privileged \
-v /dev/net:/dev/net \
quay.io/coreos/flannel:${FLANNEL_VERSION} \
--ip-masq \
--iface=${FLANNEL_IFACE}
```

The previous command should have printed a really long hash, copy this hash.
The previous command should have printed a really long hash, the container id, copy this hash.

Now get the subnet settings from flannel:

Expand Down Expand Up @@ -179,24 +203,29 @@ sudo docker run \
--privileged=true \
--pid=host \
-d \
gcr.io/google_containers/hyperkube:v${K8S_VERSION} /hyperkube kubelet --api-servers=http://localhost:8080 --v=2 --address=0.0.0.0 --enable-server --hostname-override=127.0.0.1 --config=/etc/kubernetes/manifests-multi --cluster-dns=10.0.0.10 --cluster-domain=cluster.local
gcr.io/google_containers/hyperkube:v${K8S_VERSION} \
/hyperkube kubelet \
--allow-privileged=true \
--api-servers=http://localhost:8080 \
--v=2 \
--address=0.0.0.0 \
--enable-server \
--hostname-override=127.0.0.1 \
--config=/etc/kubernetes/manifests-multi \
--containerized \
--cluster-dns=10.0.0.10 \
--cluster-domain=cluster.local
```

> Note that `--cluster-dns` and `--cluster-domain` is used to deploy dns, feel free to discard them if dns is not needed.

### Also run the service proxy

```sh
sudo docker run -d --net=host --privileged gcr.io/google_containers/hyperkube:v${K8S_VERSION} /hyperkube proxy --master=http://127.0.0.1:8080 --v=2
```

### Test it out

At this point, you should have a functioning 1-node cluster. Let's test it out!

Download the kubectl binary for `${K8S_VERSION}` (look at the URL in the following links) and make it available by editing your PATH environment variable.
([OS X](http://storage.googleapis.com/kubernetes-release/release/v1.0.7/bin/darwin/amd64/kubectl))
([linux](http://storage.googleapis.com/kubernetes-release/release/v1.0.7/bin/linux/amd64/kubectl))
([OS X](http://storage.googleapis.com/kubernetes-release/release/v1.1.3/bin/darwin/amd64/kubectl))
([linux](http://storage.googleapis.com/kubernetes-release/release/v1.1.3/bin/linux/amd64/kubectl))

For example, OS X:

Expand Down
114 changes: 57 additions & 57 deletions docs/getting-started-guides/docker-multinode/master.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.

# A scripts to install k8s worker node.
# Author @wizard_cxy @reouser
# A script to setup the k8s master in docker containers.
# Authors @wizard_cxy @resouer

set -e

Expand All @@ -26,12 +26,10 @@ if ( ! ps -ef | grep "/usr/bin/docker" | grep -v 'grep' &> /dev/null ); then
fi

# Make sure k8s version env is properly set
if [ -z ${K8S_VERSION} ]; then
K8S_VERSION="1.0.7"
echo "K8S_VERSION is not set, using default: ${K8S_VERSION}"
else
echo "k8s version is set to: ${K8S_VERSION}"
fi
K8S_VERSION=${K8S_VERSION:-"1.1.3"}
ETCD_VERSION=${ETCD_VERSION:-"2.2.1"}
FLANNEL_VERSION=${FLANNEL_VERSION:-"0.5.5"}
FLANNEL_IFACE=${FLANNEL_IFACE:-"eth0"}

# Run as root
if [ "$(id -u)" != "0" ]; then
Expand All @@ -41,12 +39,15 @@ fi

# Make sure master ip is properly set
if [ -z ${MASTER_IP} ]; then
echo "Please export MASTER_IP in your env"
exit 1
else
echo "k8s master is set to: ${MASTER_IP}"
MASTER_IP=$(hostname -I | awk '{print $1}')
fi

echo "K8S_VERSION is set to: ${K8S_VERSION}"
echo "ETCD_VERSION is set to: ${ETCD_VERSION}"
echo "FLANNEL_VERSION is set to: ${FLANNEL_VERSION}"
echo "FLANNEL_IFACE is set to: ${FLANNEL_IFACE}"
echo "MASTER_IP is set to: ${MASTER_IP}"

# Check if a command is valid
command_exists() {
command -v "$@" > /dev/null 2>&1
Expand Down Expand Up @@ -95,16 +96,17 @@ detect_lsb() {


# Start the bootstrap daemon
# TODO: do not start docker-bootstrap if it's already running
bootstrap_daemon() {
sudo -b docker -d \
-H unix:///var/run/docker-bootstrap.sock \
-p /var/run/docker-bootstrap.pid \
--iptables=false \
--ip-masq=false \
--bridge=none \
--graph=/var/lib/docker-bootstrap \
2> /var/log/docker-bootstrap.log \
1> /dev/null
docker -d \
-H unix:///var/run/docker-bootstrap.sock \
-p /var/run/docker-bootstrap.pid \
--iptables=false \
--ip-masq=false \
--bridge=none \
--graph=/var/lib/docker-bootstrap \
2> /var/log/docker-bootstrap.log \
1> /dev/null &

sleep 5
}
Expand All @@ -115,34 +117,34 @@ DOCKER_CONF=""
start_k8s(){
# Start etcd
docker -H unix:///var/run/docker-bootstrap.sock run \
--restart=always \
--net=host \
-d \
gcr.io/google_containers/etcd:2.2.1 \
/usr/local/bin/etcd \
--listen-client-urls=http://127.0.0.1:4001,http://${MASTER_IP}:4001 \
--advertise-client-urls=http://${MASTER_IP}:4001 \
--data-dir=/var/etcd/data
--restart=always \
--net=host \
-d \
gcr.io/google_containers/etcd:${ETCD_VERSION} \
/usr/local/bin/etcd \
--listen-client-urls=http://127.0.0.1:4001,http://${MASTER_IP}:4001 \
--advertise-client-urls=http://${MASTER_IP}:4001 \
--data-dir=/var/etcd/data

sleep 5
# Set flannel net config
docker -H unix:///var/run/docker-bootstrap.sock run \
--net=host gcr.io/google_containers/etcd:2.2.1 \
etcdctl \
set /coreos.com/network/config \
'{ "Network": "10.1.0.0/16", "Backend": {"Type": "vxlan"}}'
--net=host gcr.io/google_containers/etcd:${ETCD_VERSION} \
etcdctl \
set /coreos.com/network/config \
'{ "Network": "10.1.0.0/16", "Backend": {"Type": "vxlan"}}'

# iface may change to a private network interface, eth0 is for default
flannelCID=$(docker -H unix:///var/run/docker-bootstrap.sock run \
--restart=always \
-d \
--net=host \
--privileged \
-v /dev/net:/dev/net \
quay.io/coreos/flannel:0.5.5 \
/opt/bin/flanneld \
--ip-masq \
-iface="eth0")
--restart=always \
-d \
--net=host \
--privileged \
-v /dev/net:/dev/net \
quay.io/coreos/flannel:${FLANNEL_VERSION} \
/opt/bin/flanneld \
--ip-masq \
--iface="${FLANNEL_IFACE}")

sleep 8

Expand All @@ -155,13 +157,13 @@ start_k8s(){
case "${lsb_dist}" in
amzn)
DOCKER_CONF="/etc/sysconfig/docker"
echo "OPTIONS=\"\$OPTIONS --mtu=${FLANNEL_MTU} --bip=${FLANNEL_SUBNET}\"" | sudo tee -a ${DOCKER_CONF}
echo "OPTIONS=\"\$OPTIONS --mtu=${FLANNEL_MTU} --bip=${FLANNEL_SUBNET}\"" | tee -a ${DOCKER_CONF}
ifconfig docker0 down
yum -y -q install bridge-utils && brctl delbr docker0 && service docker restart
;;
centos)
DOCKER_CONF="/etc/sysconfig/docker"
echo "OPTIONS=\"\$OPTIONS --mtu=${FLANNEL_MTU} --bip=${FLANNEL_SUBNET}\"" | sudo tee -a ${DOCKER_CONF}
echo "OPTIONS=\"\$OPTIONS --mtu=${FLANNEL_MTU} --bip=${FLANNEL_SUBNET}\"" | tee -a ${DOCKER_CONF}
if ! command_exists ifconfig; then
yum -y -q install net-tools
fi
Expand All @@ -170,7 +172,7 @@ start_k8s(){
;;
ubuntu|debian)
DOCKER_CONF="/etc/default/docker"
echo "DOCKER_OPTS=\"\$DOCKER_OPTS --mtu=${FLANNEL_MTU} --bip=${FLANNEL_SUBNET}\"" | sudo tee -a ${DOCKER_CONF}
echo "DOCKER_OPTS=\"\$DOCKER_OPTS --mtu=${FLANNEL_MTU} --bip=${FLANNEL_SUBNET}\"" | tee -a ${DOCKER_CONF}
ifconfig docker0 down
apt-get install bridge-utils
brctl delbr docker0
Expand All @@ -190,7 +192,7 @@ start_k8s(){
# sleep a little bit
sleep 5

# Start kubelet & proxy, then start master components as pods
# Start kubelet and then start master components as pods
docker run \
--net=host \
--pid=host \
Expand All @@ -205,18 +207,16 @@ start_k8s(){
-v /var/lib/kubelet/:/var/lib/kubelet:rw \
gcr.io/google_containers/hyperkube:v${K8S_VERSION} \
/hyperkube kubelet \
--v=2 --address=0.0.0.0 --enable-server \
--config=/etc/kubernetes/manifests-multi \
--cluster-dns=10.0.0.10 \
--cluster-domain=cluster.local \
--containerized
--address=0.0.0.0 \
--allow-privileged=true \
--enable-server \
--api-servers=http://localhost:8080 \
--config=/etc/kubernetes/manifests-multi \
--cluster-dns=10.0.0.10 \
--cluster-domain=cluster.local \
--containerized \
--v=2

docker run \
-d \
--net=host \
--privileged \
gcr.io/google_containers/hyperkube:v${K8S_VERSION} \
/hyperkube proxy --master=http://127.0.0.1:8080 --v=2
}

echo "Detecting your OS distro ..."
Expand Down