Skip to content
This repository has been archived by the owner on Jun 29, 2022. It is now read-only.

Latest commit

聽

History

History
300 lines (207 loc) 路 8.64 KB

baremetal.md

File metadata and controls

300 lines (207 loc) 路 8.64 KB
Raw
title weight
Lokomotive Bare Metal quickstart guide
10

Introduction

This quickstart guide walks through the steps needed to create a Lokomotive cluster on bare metal with Flatcar Container Linux utilizing PXE.

We recommend you to check out Racker for an integrated solution based on the Lokomotive bare metal platform.

By the end of this guide, you'll have a working Kubernetes cluster with 1 controller node and 2 worker nodes.

Requirements

  • Basic understanding of Kubernetes concepts.
  • Terraform v0.13.x installed locally.
  • Machines with at least 2.5GB RAM, 30GB disk, PXE-enabled NIC and IPMI.
  • PXE-enabled network boot environment.
  • Matchbox v0.6+ deployment with API enabled.
  • Matchbox credentials client.crt, client.key, ca.crt.
  • An SSH key pair for management access.
  • kubectl installed locally to access the Kubernetes cluster.

Note that without a proper pxe_commands value the machines should only be powered on manually after starting the installation, see below.

Steps

Step 1: Install lokoctl

lokoctl is a command-line interface for Lokomotive.

To install lokoctl, follow the instructions in the lokoctl installation guide.

Step 2: Set up a working directory

It's better to start fresh in a new working directory, as the state of the cluster is stored in this directory.

This also makes the cleanup task easier.

mkdir -p lokomotive-infra/mybaremetalcluster
cd lokomotive-infra/mybaremetalcluster

Step 3: Machines, DNS and Matchbox set up

Machines

Mac addresses collected from each machine.

For machines with multiple PXE-enabled NICs, pick one of the MAC addresses. MAC addresses will be used to match machines to profiles during network boot.

Example:

52:54:00:a1:9c:ae (node1)
52:54:00:b2:2f:86 (node2)
52:54:00:c3:61:77 (node3)

DNS

Create DNS A (or AAAA) record for each node's default interface.

Cluster nodes will be configured to refer to the control plane and themselves by these fully qualified names and they will be used in generated TLS certificates.

Example:

node1.example.com (node1)
node2.example.com (node2)
node3.example.com (node3)

Matchbox

One of the requirements is to have Matchbox with TLS enabled deployed.

Verify the Matchbox read-only HTTP endpoints are accessible.

curl http://matchbox.example.com:8080
matchbox

Verify your TLS client certificate and key can be used to access the Matchbox API.

openssl s_client -connect matchbox.example.com:8081 \
  -CAfile /path/to/matchbox/ca.crt \
  -cert /path/to/matchbox/client.crt \
  -key /path/to/matchbox/client.key

Step 4: Define Cluster Configuration

To create a Lokomotive cluster, we need to define a configuration.

Create a file with the extension .lokocfg with the contents below:

# baremetalcluster.lokocfg
cluster "bare-metal" {
  # Change the location where lokoctl stores the cluster assets.
  asset_dir = "./lokomotive-assets"

  # Cluster name.
  cluster_name = "baremetalcluster"

  # SSH Public keys.
  ssh_pubkeys = [
    "ssh-rsa AAAAB3Nz...",
  ]
  # Whether the operating system should PXE boot and install from matchbox /assets cache.
  cached_install = "true"

  # Matchbox CA crt path.
  matchbox_ca_path = pathexpand("/path/to/matchbox/ca.crt")

  # Matchbox client crt path.
  matchbox_client_cert_path = pathexpand("/path/to/matchbox/client.crt")

  # Matchbox client key path.
  matchbox_client_key_path = pathexpand("/path/to/matchbox/client.key")

  # Matchbox https endpoint.
  matchbox_endpoint = "matchbox.example.com:8081"

  # Matchbox HTTP read-only endpoint.
  matchbox_http_endpoint = "http://matchbox.example.com:8080"

  # Domain name.
  k8s_domain_name = "example.com"

  # FQDN of controller nodes.
  controller_domains = [
    "node1.example.com",
  ]

  # MAC addresses of controllers.
  controller_macs = [
    "52:54:00:a1:9c:ae",
  ]

  # Names of the controller nodes.
  controller_names = [
    "node1",
  ]

  # FQDN of worker nodes.
  worker_domains = [
    "node2.example.com",
    "node3.example.com",
  ]

  # Mac addresses of worker nodes.
  worker_macs = [
    "52:54:00:b2:2f:86",
    "52:54:00:c3:61:77",
  ]

  # Names of the worker nodes.
  worker_names = [
    "node2",
    "node3",
  ]

  # Automation to force a PXE boot, a dummy sleep for now as you will do it manually.
  pxe_commands = "sleep 300"
}

Lokomotive can store Terraform state locally or remotely within an AWS S3 bucket. By default, Lokomotive stores Terraform state locally.

For advanced cluster configurations and more information refer to the Bare Metal configuration guide.

Step 5: Create Lokomotive Cluster

Add a private key corresponding to one of the public keys specified in ssh_pubkeys to your ssh-agent:

ssh-add ~/.ssh/id_rsa
ssh-add -L

Run the following command to create the cluster:

lokoctl cluster apply

Proceed to Power on the PXE machines while this loops, but only after Matchbox has the configuration ready. See the configuration reference on how to properly configure pxe_commands to automate the provisioning reliably.

Once the command finishes, your Lokomotive cluster details are stored in the path you've specified under asset_dir.

Verification

A successful installation results in the output:

module.baremetal-baremetalcluster.null_resource.bootkube-start: Still creating... [4m10s elapsed]
module.baremetal-baremetalcluster.null_resource.bootkube-start: Still creating... [4m20s elapsed]
module.baremetal-baremetalcluster.null_resource.bootkube-start: Creation complete after 4m25s [id=1122239320434737682]

Apply complete! Resources: 74 added, 0 changed, 0 destroyed.

Your configurations are stored in /home/imran/lokoctl-assets/mycluster

Now checking health and readiness of the cluster nodes ...

Node                                          Ready    Reason          Message

node1.example.com                             True     KubeletReady    kubelet is posting ready status
node2.example.com                             True     KubeletReady    kubelet is posting ready status
node3.example.com                             True     KubeletReady    kubelet is posting ready status

Success - cluster is healthy and nodes are ready!

Use the generated kubeconfig file to access the Kubernetes cluster and list nodes.

export KUBECONFIG=./lokomotive-assets/cluster-assets/auth/kubeconfig
kubectl get nodes

Using the cluster

At this point you have access to the Kubernetes cluster and can use it! If you don't have Kubernetes experience you can check out the Kubernetes Basics official documentation to learn about its usage.

Note: Lokomotive sets up a pretty restrictive Pod Security Policy that disallows running containers as root by default, check the Pod Security Policy documentation for more details.

Cleanup

To destroy the Lokomotive cluster, execute the following command:

lokoctl cluster destroy --confirm

You can safely delete the working directory created for this quickstart guide if you no longer require it.

Troubleshooting

Stuck At Copy Controller Secrets

If there is an execution error or no progress beyond the output provided below:

...
module.baremetal-baremetalcluster.null_resource.copy-controller-secrets: Still creating... (8m30s elapsed)
module.baremetal-baremetalcluster.null_resource.copy-controller-secrets: Still creating... (8m40s elapsed)
...

The error probably happens because the ssh_pubkeys provided in the configuration is missing in the ssh-agent.

To rectify the error, you need to:

  1. Follow the steps to add the SSH key to the ssh-agent.
  2. Retry Step 5.

Conclusion

After walking through this guide, you've learned how to set up a Lokomotive cluster on Bare Metal.

Next steps

You can now start deploying your workloads on the cluster.

For more information on installing supported Lokomotive components, you can visit the component configuration guides.