Homelab

On-prem Kubernetes cluster.

Requirements

Command line tools

Required

• Helm
• Kubectl
• Kubelogin

Optional

• Cilium cli
• CMctl
• Talosctl

Hardware

• Ubiquiti EdgeRouter X

  Acts as a sub-router in my network using static routes & proper firewall configuration.

• Super6c raspberry pi cluster

  6 raspberry pi CM4 nodes with a single m.2 sdd each connected to an unmanaged switch all on a single board.

• Raspberry pi 4 4GB

  Runs some docker services that cant run inside the cluster.

Setup

Kubernetes

Keep in mind that this stuff cant run inside the kubernetes cluster and requires a separate docker host.

• Sidero omni

  Alternatively you could just use Talos linux and talosctl to manage your nodes.

  Dashboard: https://ceph.rickli.cloud

  • Custom pre configured Talos linux images.
  • Cluster lifecycle management.
  • Kubernetes API access control (OAuth).

Networking

• Cilium

  Dashboard: https://hubble.rickli.cloud

  • Replaces kube-proxy.
  • Acts as LoadBalancer & assigns external ip's.
  • Networking policies.
  • Service map (hubble).

• External DNS

  Endpoint: udp://172.16.1.1

  • Resolves ingress hostnames to corresponding external virtual ip.

Certificates

• Cert manager

  • Manages tls certificates for ingress resources thru private or public key infrastructure.

Storage

• Rook ceph

  Dashboard: https://ceph.rickli.cloud

  • Provides storage for the kubernetes cluster.

Monitoring

• Netdata

  Dashboard: https://netdata.rickli.cloud

  • Monitors each node and provides a dashboard without a big configuration mess.

IP ranges

EdgeRouter

IP range: 192.168.10.0/24

kubernetes services

IP range: 172.16.0.0/16 (virtual)