Personal media server

Installation

Install helm and fluxctl

brew install helm fluxctl

Create a new VM from the k3os installation ISO.

Log in as the rancher user and execute sudo k3os install. Select "Install to disk". Select "Config with cloud-init file". Use the path

https://raw.githubusercontent.com/cjlarose/media-server/master/cloud-init-server.yaml

The VM will install k3os to disk and restart.

Stop the VM. Remove the installation media from the VM. Start it back up.

Now, let's ensure we can connect to the VM removely via kubectl.

K3OS_IP=192.168.50.143
scp rancher@"$K3OS_IP":/etc/rancher/k3s/k3s.yaml ~/.kube/k3os.yaml
ln -sfn ~/.kube/k3os.yaml ~/.kube/config
sed -i .bak 's/127.0.0.1/'"$K3OS_IP"'/g' ~/.kube/k3os.yaml

Now, execute kubectl version to make sure you can connect. All further instructions are to be executed remotely.

Label all nodes:

kubectl label nodes --all kubernetes-host=

Install Calico:

kubectl create -f https://docs.projectcalico.org/manifests/tigera-operator.yaml
cat <<EOF | kubectl create -f -
# This section includes base Calico installation configuration.
# For more information, see: https://docs.projectcalico.org/v3.15/reference/installation/api#operator.tigera.io/v1.Installation
apiVersion: operator.tigera.io/v1
kind: Installation
metadata:
  name: default
spec:
  # Configures Calico networking.
  calicoNetwork:
    # Note: The ipPools section cannot be modified post-install.
    ipPools:
    - blockSize: 26
      cidr: 10.42.0.0/16
      encapsulation: VXLANCrossSubnet
      natOutgoing: Enabled
      nodeSelector: all()
EOF

Apply Calico network policies

find ./calico -name '*.yaml' -exec calicoctl apply -f {} \;

Install MetalLB:

kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.3/manifests/namespace.yaml
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.3/manifests/metallb.yaml
kubectl -n metallb-system patch daemonset speaker --patch-file metallb/speaker-patch.yaml
kubectl -n metallb-system patch deployment controller --patch-file metallb/controller-patch.yaml
# On first install only
kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"

Configure MetalLB:

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: config
data:
  config: |
    address-pools:
    - name: default
      protocol: layer2
      addresses:
      - 172.16.20.17/32
      - 172.16.20.23/32
EOF

Install cert-manager CRDs:

kubectl apply --validate=false \
-f https://github.com/jetstack/cert-manager/releases/download/v1.1.0/cert-manager.crds.yaml

Install sealed-secrets CRDs:

kubectl apply -f https://raw.githubusercontent.com/bitnami-labs/sealed-secrets/v0.15.0/helm/sealed-secrets/crds/sealedsecret-crd.yaml

Install flux and the helm operator:

helm repo add fluxcd https://charts.fluxcd.io

kubectl create namespace flux

helm upgrade -i flux-homelab fluxcd/flux \
--set git.url=git@github.com:cjlarose/homelab \
--set git.path="cert-manager\,configmaps\,namespaces\,releases\,secrets\,workloads" \
--set syncGarbageCollection.enabled=true \
--set registry.excludeImage='*' \
--version 1.8.0 \
--namespace flux

helm upgrade -i helm-operator fluxcd/helm-operator \
--set helm.versions=v3 \
--set allowNamespace=media \
--namespace media

helm upgrade -i helm-operator-cert-manager fluxcd/helm-operator \
--set helm.versions=v3 \
--set allowNamespace=cert-manager \
--namespace cert-manager

Wait until the containers come up. Then, use fluxctl to get the SSH public key for deployments

fluxctl identity --k8s-fwd-ns flux

Add that as a "deploy key" for this repository. Allow write access.

Deploying changes

Just push to this repository. flux will update resources in the cluster automatically.

If you're impatient, trigger changes manually with

fluxctl --k8s-fwd-ns=flux sync

Update the default service account in the media namespace to use a specific imagePullSecrets:

kubectl -n media patch serviceaccount default -p '{"imagePullSecrets": [{"name": "docker-hub"}]}'

Logs

To monitor flux logs

kubectl -n flux logs deployment/flux-homelab -f

To monitor helm operator logs (media namespace)

kubectl -n media logs deployment/helm-operator -f

To monitor helm operator logs (cert-manager namespace)

kubectl -n cert-manager logs deployment/helm-operator-cert-manager -f