Turing Pi 2 Kubernetes Cluster

Flux driven bare metal kubernetes cluster living on the edge.

Getting Started

Turing Pi

A compact AI & edge computing cluster.

mindmap
  root(clusters/local)
    apps
      home
        homebridge
        zigbee2MQTT
    charts
      cert manager
      cilium
      cloudflare tunnel
      grafana
      MQTT
      node feature discovery
      nvidia device plugin
      postgres
      redpanda
      vault
    manifests
      GatewayClass
      Gateway
      RuntimeClass
      StorageClass
    notifications
      slack
      webhooks

Build

• Densium APU 4L
• Noctua NF-A14 ULN 140mm
• Pico PSU 160w
• 3x Turing RK1 8 cores / 32 GB / 6 TOPS
• 1x Nvidia Jetson Orin NX 8 cores / 16 GB / 100 TOPS
• 4x Samsung 970 EVO Plus NVMe M.2 SSD 250GB

32 cores / 112 GB / 118 TOPS

1TB storage

https://turingpi.com/

Flashing Nodes

ssh turingpi
cd /mnt/sdcard

curl -k https://firmware.turingpi.com/turing-rk1/ubuntu_22.04_rockchip_linux/v1.32/ubuntu-22.04.3-preinstalled-desktop-arm64-turing-rk1_v1.32.img.xz -o ubuntu-22.04.3-preinstalled-desktop-arm64-turing-rk1_v1.32.img.xz
xz -d  ubuntu-22.04.3-preinstalled-desktop-arm64-turing-rk1_v1.32.img.xz

tpi flash -n N -i /mnt/sdcard/ubuntu-22.04.3-preinstalled-server-arm64-turing-rk1_v1.32.img

Kubernetes (via k3s)

An open-source system for automating deployment, scaling, and management of containerized applications.

https://docs.k3s.io/

Flux

Used to pull repository changes into kubernetes clusters.

https://fluxcd.io/

Cilium

Used for providing, securing, and observing network connectivity between workloads, powered by eBPF.

https://cilium.io/

Brew

The Missing Package Manager for macOS (or Linux).

https://brew.sh

This repo includes a collection of dependencies to install:

brew bundle

Usage

Bootstrap

When spinning up the cluster for the first time, there are 3 primary steps.

1. Install k3s

   https://docs.turingpi.com/docs/turing-pi2-kubernetes-installation https://docs.k3s.io/installation/configuration https://docs.k3s.io/installation/network-options

   # Controller
   
   export K3S_TOKEN=$(uuidgen)
   export CONTROLLER_IP=192.168.4.78
   
   curl -sfL https://get.k3s.io | sh -s - \
   --write-kubeconfig-mode 644 \
   --token $K3S_TOKEN \
   --node-ip $CONTROLLER_IP \
   --flannel-backend=none \
   --disable-cloud-controller \
   --disable-kube-proxy \
   --disable-network-policy \
   --disable local-storage \
   --disable servicelb \
   --disable traefik
   
   # Workers
   
   export K3S_TOKEN=
   export K3S_URL=https://$CONTROLLER_IP:6443
   
   curl -sfL https://get.k3s.io | sh -
   
   # optional check to ensure a node is fully operational
   k3s check-config

2. Install cilium

   https://docs.cilium.io/en/stable/gettingstarted/k8s-install-default/

   • System Requirements
   • Rebuilding the Linux Kernel

   cilium install --version 1.15.4
   cilium status --wait

3. Bootstrap flux

   https://fluxcd.io/flux/installation/bootstrap/github/

   flux bootstrap github \
     --components-extra=image-reflector-controller,image-automation-controller \
     --owner=dudo \
     --repository=turing-pi \
     --private=false \
     --personal=true \
     --path=clusters/overlays/local

kubectl

https://kubernetes.io/docs/reference/kubectl/cheatsheet/

kubectl get GitRepository -n flux-system
kubectl get Kustomization -n flux-system
kubectl get HelmRelease -n blue
kubectl logs -n flux-system deploy/image-automation-controller

kubectl debug POD -it --image=busybox

flux

https://fluxcd.io/flux/cmd/

flux get all -A

flux suspend image update my-service
flux resume image update my-service

flux reconcile source git flux-system
flux reconcile kustomization flux-system
flux reconcile kustomization charts

kubeseal

https://github.com/bitnami-labs/sealed-secrets

encoded_string=$(echo -n "This is a string" | base64)
encoded_string=$(base64 <<EOF
  This is a
  multi-line string
  that I want to encode.
EOF
)

kubeseal --format=yaml <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: mysecret
  namespace: whatever
data:
  my.file: ${encoded_string}
EOF

Pertinent Sections

• Apps
• Charts
• Clusters
• Manifests
• Notifications