Skip to content
This repository has been archived by the owner on Jun 29, 2022. It is now read-only.

Add the kvm-libvirt platform to lokoctl #810

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

Add the kvm-libvirt platform to lokoctl #810

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
9edcfe6
Add the kvm-libvirt platform to lokoctl
pothos Aug 12, 2020
9b28bd4
terraform: update terraform providers to new format
ipochi Oct 21, 2020
08a65cb
platform/kvmlibvirt: add charts to be installed
ipochi Oct 21, 2020
bd7ce15
kvmlibvirt: enable tls bootstrap
ipochi Oct 21, 2020
e052b80
lint: fix lint issues
ipochi Oct 21, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ Follow one of the quickstart guides for the supported platforms:
* [Packet quickstart](docs/quickstarts/packet.md)
* [AWS quickstart](docs/quickstarts/aws.md)
* [Bare metal quickstart](docs/quickstarts/baremetal.md)
* [KVM libvirt quickstart](docs/quickstarts/kvm-libvirt.md)

## Documentation

Expand Down
5 changes: 5 additions & 0 deletions assets/terraform-modules/kvm-libvirt/flatcar-linux/kubernetes/bootkube.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,11 @@ module "bootkube" {
enable_aggregation = var.enable_aggregation
encrypt_pod_traffic = var.encrypt_pod_traffic

# Disable the self hosted kubelet.
disable_self_hosted_kubelet = var.disable_self_hosted_kubelet
# Extra flags to API server.
kube_apiserver_extra_flags = var.kube_apiserver_extra_flags

certs_validity_period_hours = var.certs_validity_period_hours
}

Expand Down
2 changes: 1 addition & 1 deletion assets/terraform-modules/kvm-libvirt/flatcar-linux/kubernetes/controllers.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ resource "libvirt_pool" "volumetmp" {

resource "libvirt_volume" "base" {
name = "${var.cluster_name}-base"
source = var.os_image_unpacked
source = var.os_image
pool = libvirt_pool.volumetmp.name
format = "qcow2"
}
Expand Down
12 changes: 0 additions & 12 deletions assets/terraform-modules/kvm-libvirt/flatcar-linux/kubernetes/outputs.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,6 @@ output "kubeconfig" {
value = module.bootkube.kubeconfig-kubelet
}

output "machine_domain" {
value = var.machine_domain
}

output "cluster_name" {
value = var.cluster_name
}

output "ssh_keys" {
value = var.ssh_keys
}

output "libvirtpool" {
value = libvirt_pool.volumetmp.name
}
Expand Down
13 changes: 12 additions & 1 deletion assets/terraform-modules/kvm-libvirt/flatcar-linux/kubernetes/variables.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ variable "cluster_name" {
}

# Nodes
variable "os_image_unpacked" {
variable "os_image" {
type = string
description = "Path to unpacked Flatcar Container Linux image flatcar_production_qemu_image.img (probably after a qemu-img resize IMG +5G)"
}
Expand Down Expand Up @@ -105,6 +105,17 @@ variable "enable_aggregation" {
default = true
}

variable "kube_apiserver_extra_flags" {
description = "Extra flags passed to self-hosted kube-apiserver."
type = list(string)
default = []
}

variable "disable_self_hosted_kubelet" {
description = "Disable the self hosted kubelet installed by default"
type = bool
}

# Certificates

variable "certs_validity_period_hours" {
Expand Down
11 changes: 11 additions & 0 deletions ci/kvm-libvirt/kvm-libvirt-cluster.lokocfg.envsubst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
cluster "kvm-libvirt" {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have a CI pipeline for this platform yet, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There was one which shared the baremetal CI setup. The code was probably removed but can be recovered from git.

asset_dir = pathexpand("~/lokoctl-assets")
ssh_pubkeys = [file(pathexpand("~/.ssh/id_rsa.pub"))]
cluster_name = "vmcluster"
machine_domain = "vmcluster.k8s"
os_image = "file:///var/tmp/flatcar_production_qemu_image.img"

worker_pool "one" {
count = 2
}
}
1 change: 1 addition & 0 deletions cli/cmd/root.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import (
_ "github.com/kinvolk/lokomotive/pkg/platform/aks"
_ "github.com/kinvolk/lokomotive/pkg/platform/aws"
_ "github.com/kinvolk/lokomotive/pkg/platform/baremetal"
_ "github.com/kinvolk/lokomotive/pkg/platform/kvmlibvirt"
_ "github.com/kinvolk/lokomotive/pkg/platform/packet"

// Register backends by adding an anonymous import.
Expand Down
142 changes: 142 additions & 0 deletions docs/configuration-reference/platforms/kvm-libvirt.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
# Lokomotive KVM libvirt configuration reference

## Contents

* [Introduction](#introduction)
* [Prerequisites](#prerequisites)
* [Configuration](#configuration)
* [Attribute reference](#attribute-reference)
* [Applying](#applying)
* [Destroying](#destroying)

## Introduction

This configuration reference provides information on configuring a Lokomotive cluster on KVM libvirt VMs
with all the configuration options available to the user.

## Prerequisites

* Terraform providers and libvirt setup from the [quickstart guide](../../quickstarts/kvm-libvirt.md)
* `lokoctl` [installed locally.](../../installer/lokoctl.md)
* `kubectl` installed locally to access the Kubernetes cluster.

### Configuration

To create a Lokomotive cluster, we need to define a configuration.

Example configuration file:

```tf

cluster "kvm-libvirt" {

asset_dir = pathexpand("./assets")

cluster_name = "vmcluster"

machine_domain = "vmcluster.k8s"

os_image = "file:///home/myuser/Downloads/flatcar_production_qemu_image.img"

ssh_pubkeys = [file(pathexpand("~/.ssh/id_rsa.pub"))]

controller_count = 1

node_ip_pool = "192.168.192.0/24"

disable_self_hosted_kubelet = false

kube_apiserver_extra_flags = []

controller_virtual_cpus = 1
controller_virtual_memory = 2048

controller_clc_snippets = []

network_mtu = 1480
network_ip_autodetection_method = "first-found"

pod_cidr = "10.1.0.0/16"
service_cidr = "10.2.0.0/16"

cluster_domain_suffix = "cluster.local"

enable_reporting = false
enable_aggregation = true

certs_validity_period_hours = 8760

worker_pool "worker-pool-1" {
count = 2

virtual_cpus = 1
virtual_memory = 2048

labels = "foo=oof,bar=,baz=zab"

clc_snippets = [
<<EOF
systemd:
units:
- name: helloworld.service
dropins:
- name: 10-helloworld.conf
contents: |
[Install]
WantedBy=multi-user.target
EOF
,
]

}
}
```


## Attribute reference

| Argument | Description | Default | Type | Required |
|---------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------:|:------------:|:--------:|
| `asset_dir` | Location where Lokomotive stores cluster assets. | - | string | true |
| `certs_validity_period_hours` | Validity of all the certificates in hours. | 8760 | number | false |
| `cluster_domain_suffix` | Cluster's DNS domain. | "cluster.local" | string | false |
| `cluster_name` | Name of the cluster. | - | string | true |
| `controller_clc_snippets` | Controller Flatcar Container Linux Config snippets. | [] | list(string) | false |
| `controller_count` | Number of controller nodes. | 1 | number | false |
| `controller_virtual_cpus` | Number of virtual CPUs for the controller VMs. | 1 | int | false |
| `controller_virtual_memory` | Virtual RAM in MB for the controller VMs. | 2048 | int | false |
| `disable_self_hosted_kubelet` | Disable self-hosting the kubelet as pod on the cluster. | false | bool | false |
| `enable_aggregation` | Enable the Kubernetes Aggregation Layer. | true | bool | false |
| `enable_reporting` | Enables usage or analytics reporting to upstream. | false | bool | false |
| `kube_apiserver_extra_flags` | Extra flags to pass to the kube-apiserver binary. | [] | list(string) | false |
| `machine_domain` | DNS zone of the cluster, used by nodes to find each other as HOSTNAME.machine_domain. | - | string | true |
| `network_mtu` | CNI interface MTU | 1480 | number | false |
| `node_ip_pool` | Unique VM IP CIDR. | "192.168.192.0/24" | string | false |
| `os_image` | Path to unpacked Flatcar Container Linux image flatcar_production_qemu_image.img (probably after a qemu-img resize IMG +5G). | - | string | true |
| `pod_cidr` | CIDR IPv4 range to assign Kubernetes pods. | "10.2.0.0/16" | string | false |
| `service_cidr` | CIDR IPv4 range to assign Kubernetes services. | "10.3.0.0/16" | string | false |
| `ssh_pubkeys` | List of SSH public keys for user `core`. Each element must be specified in a valid OpenSSH public key format, as defined in RFC 4253 Section 6.6, e.g. "ssh-rsa AAAAB3N...". | - | list(string) | true |
| `worker_pool.clc_snippets` | Flatcar Container Linux Config snippets for nodes in the worker pool. | [] | list(string) | false |
| `worker_pool` | Configuration block for worker pools. There can be more than one. | - | list(object) | true |
| `worker_pool.count` | Number of workers in the worker pool. Can be changed afterwards to add or delete workers. | 1 | number | true |
| `worker_pool.labels` | Custom labels to assign to worker nodes. | - | string | false |
| `worker_pool.virtual_cpus` | List of tags that will be propagated to nodes in the worker pool. | - | map(string) | false |
| `worker_pool.virtual_memory` | Disable BGP on nodes. Nodes won't be able to connect to Packet BGP peers. | false | bool | false |


## Applying

To create the cluster, execute the following command:

```console
lokoctl cluster apply
```

## Destroying

To destroy the Lokomotive cluster, execute the following command:

```console
lokoctl cluster destroy --confirm
```