Skip to content

Latest commit

 

History

History
183 lines (130 loc) · 5.74 KB

getting-started-rkt.md

File metadata and controls

183 lines (130 loc) · 5.74 KB
Raw

Getting started with rkt

In this tutorial, we'll run matchbox on your Linux machine with rkt and CNI to network boot and provision a cluster of QEMU/KVM CoreOS machines locally. You'll be able to create Kubernetes clustes, etcd3 clusters, and test network setups.

Note: To provision physical machines, see network setup and deployment.

Requirements

Install rkt 1.12.0 or higher (example script) and setup rkt privilege separation.

Next, install the package dependencies.

# Fedora
$ sudo dnf install virt-install virt-manager

# Debian/Ubuntu
$ sudo apt-get install virt-manager virtinst qemu-kvm systemd-container

Note: rkt does not yet integrate with SELinux on Fedora. As a workaround, temporarily set enforcement to permissive if you are comfortable (sudo setenforce Permissive). Check the rkt distribution notes or see the tracking issue.

Clone the matchbox source which contains the examples and scripts.

$ git clone https://github.com/coreos/matchbox.git
$ cd matchbox

Download CoreOS image assets referenced by the etcd example to examples/assets.

$ ./scripts/get-coreos stable 1353.7.0 ./examples/assets

Network

Define the metal0 virtual bridge with CNI.

sudo mkdir -p /etc/rkt/net.d
sudo bash -c 'cat > /etc/rkt/net.d/20-metal.conf << EOF
{
  "name": "metal0",
  "type": "bridge",
  "bridge": "metal0",
  "isGateway": true,
  "ipMasq": true,
  "ipam": {
    "type": "host-local",
    "subnet": "172.18.0.0/24",
    "routes" : [ { "dst" : "0.0.0.0/0" } ]
   }
}
EOF'

On Fedora, add the metal0 interface to the trusted zone in your firewall configuration.

$ sudo firewall-cmd --add-interface=metal0 --zone=trusted
$ sudo firewall-cmd --add-interface=metal0 --zone=trusted --permanent

For development convenience, you may wish to add /etc/hosts entries for nodes to refer to them by name.

# /etc/hosts
...
172.18.0.21 node1.example.com
172.18.0.22 node2.example.com
172.18.0.23 node3.example.com

Containers

Run the matchbox and dnsmasq services on the metal0 bridge. dnsmasq will run DHCP, DNS, and TFTP services to create a suitable network boot environment. matchbox will serve provisioning configs to machines on the network which attempt to PXE boot.

The devnet wrapper script rkt runs matchbox and dnsmasq in systemd transient units. Create can take the name of any example cluster in examples.

$ sudo ./scripts/devnet create etcd3

Inspect the journal logs or check the status of the systemd services.

$ sudo ./scripts/devnet status
$ journalctl -f -u dev-matchbox
$ journalctl -f -u dev-dnsmasq

Take a look at the etcd3 groups to get an idea of how machines are mapped to Profiles. Explore some endpoints exposed by the service, say for QEMU/KVM node1.

Manual

If you prefer to start the containers yourself, instead of using devnet,

sudo rkt run --net=metal0:IP=172.18.0.2 \
  --mount volume=data,target=/var/lib/matchbox \
  --volume data,kind=host,source=$PWD/examples \
  --mount volume=groups,target=/var/lib/matchbox/groups \
  --volume groups,kind=host,source=$PWD/examples/groups/etcd3 \
  quay.io/coreos/matchbox:v0.6.0 -- -address=0.0.0.0:8080 -log-level=debug
sudo rkt run --net=metal0:IP=172.18.0.3 \
  --dns=host \
  --mount volume=config,target=/etc/dnsmasq.conf \
  --volume config,kind=host,source=$PWD/contrib/dnsmasq/metal0.conf \
  quay.io/coreos/dnsmasq:v0.4.0 \
  --caps-retain=CAP_NET_ADMIN,CAP_NET_BIND_SERVICE,CAP_SETGID,CAP_SETUID,CAP_NET_RAW

If you get an error about the IP assignment, stop old pods and run garbage collection.

$ sudo rkt gc --grace-period=0

Client VMs

Create QEMU/KVM VMs which have known hardware attributes. The nodes will be attached to the metal0 bridge, where your pods run.

$ sudo ./scripts/libvirt create

You can connect to the serial console of any node. If you provisioned nodes with an SSH key, you can SSH after bring-up.

$ sudo virsh console node1

You can also use virt-manager to watch the console.

$ sudo virt-manager

Use the wrapper script to act on all nodes.

$ sudo ./scripts/libvirt [start|reboot|shutdown|poweroff|destroy]

Verify

The VMs should network boot and provision themselves into a three node etcd3 cluster, with other nodes behaving as etcd3 gateways.

The example profile added autologin so you can verify that etcd3 works between nodes.

$ systemctl status etcd-member
$ ETCDCTL_API=3
$ etcdctl set /message hello
$ etcdctl get /message

Clean up

Clean up the systemd units running matchbox and dnsmasq.

$ sudo ./scripts/devnet destroy

Clean up VM machines.

$ sudo ./scripts/libvirt destroy

Press ^] three times to stop any rkt pod.

Going further

Learn more about matchbox or explore the other example clusters. Try the k8s example to produce a TLS-authenticated Kubernetes cluster you can access locally with kubectl.