Install helm
and fluxctl
brew install helm fluxctl
Create a new VM from the k3os
installation ISO.
Log in as the rancher
user and execute sudo k3os install
. Select "Install to disk". Select "Config with cloud-init file". Use the path
https://raw.githubusercontent.com/cjlarose/media-server/master/cloud-init-server.yaml
The VM will install k3os
to disk and restart.
Stop the VM. Remove the installation media from the VM. Start it back up.
Now, let's ensure we can connect to the VM removely via kubectl
.
K3OS_IP=192.168.50.143
scp rancher@"$K3OS_IP":/etc/rancher/k3s/k3s.yaml ~/.kube/k3os.yaml
ln -sfn ~/.kube/k3os.yaml ~/.kube/config
sed -i .bak 's/127.0.0.1/'"$K3OS_IP"'/g' ~/.kube/k3os.yaml
Now, execute kubectl version
to make sure you can connect. All further instructions are to be executed remotely.
Label all nodes:
kubectl label nodes --all kubernetes-host=
Install Calico:
kubectl create -f https://docs.projectcalico.org/manifests/tigera-operator.yaml
cat <<EOF | kubectl create -f -
# This section includes base Calico installation configuration.
# For more information, see: https://docs.projectcalico.org/v3.15/reference/installation/api#operator.tigera.io/v1.Installation
apiVersion: operator.tigera.io/v1
kind: Installation
metadata:
name: default
spec:
# Configures Calico networking.
calicoNetwork:
# Note: The ipPools section cannot be modified post-install.
ipPools:
- blockSize: 26
cidr: 10.42.0.0/16
encapsulation: VXLANCrossSubnet
natOutgoing: Enabled
nodeSelector: all()
EOF
Apply Calico network policies
find ./calico -name '*.yaml' -exec calicoctl apply -f {} \;
Install MetalLB:
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.3/manifests/namespace.yaml
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.3/manifests/metallb.yaml
kubectl -n metallb-system patch daemonset speaker --patch-file metallb/speaker-patch.yaml
kubectl -n metallb-system patch deployment controller --patch-file metallb/controller-patch.yaml
# On first install only
kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"
Configure MetalLB:
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
namespace: metallb-system
name: config
data:
config: |
address-pools:
- name: default
protocol: layer2
addresses:
- 172.16.20.17/32
- 172.16.20.23/32
EOF
Install cert-manager CRDs:
kubectl apply --validate=false \
-f https://github.com/jetstack/cert-manager/releases/download/v1.1.0/cert-manager.crds.yaml
Install sealed-secrets CRDs:
kubectl apply -f https://raw.githubusercontent.com/bitnami-labs/sealed-secrets/v0.15.0/helm/sealed-secrets/crds/sealedsecret-crd.yaml
Install flux and the helm operator:
helm repo add fluxcd https://charts.fluxcd.io
kubectl create namespace flux
helm upgrade -i flux-homelab fluxcd/flux \
--set git.url=git@github.com:cjlarose/homelab \
--set git.path="cert-manager\,configmaps\,namespaces\,releases\,secrets\,workloads" \
--set syncGarbageCollection.enabled=true \
--set registry.excludeImage='*' \
--version 1.8.0 \
--namespace flux
helm upgrade -i helm-operator fluxcd/helm-operator \
--set helm.versions=v3 \
--set allowNamespace=media \
--namespace media
helm upgrade -i helm-operator-cert-manager fluxcd/helm-operator \
--set helm.versions=v3 \
--set allowNamespace=cert-manager \
--namespace cert-manager
Wait until the containers come up. Then, use fluxctl
to get the SSH public key for deployments
fluxctl identity --k8s-fwd-ns flux
Add that as a "deploy key" for this repository. Allow write access.
Just push to this repository. flux
will update resources in the cluster automatically.
If you're impatient, trigger changes manually with
fluxctl --k8s-fwd-ns=flux sync
Update the default service account in the media
namespace to use a specific imagePullSecrets
:
kubectl -n media patch serviceaccount default -p '{"imagePullSecrets": [{"name": "docker-hub"}]}'
To monitor flux logs
kubectl -n flux logs deployment/flux-homelab -f
To monitor helm operator logs (media
namespace)
kubectl -n media logs deployment/helm-operator -f
To monitor helm operator logs (cert-manager
namespace)
kubectl -n cert-manager logs deployment/helm-operator-cert-manager -f