home-server

Proxmox Virtual Environment (Proxmox VE) for running VMs and Kubernetes Cluster provisioned using Terraform and Ansible.

Table of Contents

• Diagram
• Product Structure
• OPNsense Router
• Jumphost VM
• Kubernetes Cluster
  • Authentication
  • Service Mesh / API Gateway
• Terraform Diagram

Home server Diagram

Project Structure

.
├── ansible                             # ansible playbook
├── kubernetes                          # kubernetes/ helm chart manifests
├── packer                              # packer vm templates
└── terraform                           # terraform iac
    └── _talos                          # talos kubernetes cluster config
    └── modules                         
        └── kubernetes                  # kubernetes cluster essentials
        └── kubernetes-addons           # kubernetes addons
        └── talos-k8s                   # talos kubernetes vm
        └── talos-patch                 # talos kubernetes vm patch
        └── vm                          # jumphost / router vm
        └── vm-templates                # vm templates

OPNsense Router

OPNsense router with zerotier plugin for VPC

Default Credentials

• username: root
• password: opnsense

Network Interface

Interface	Network	Description	IP
LAN	vtnet0	Linux Bridge	192.168.0.1/24
OPT	vtnet1	VM Network	172.16.0.0/16
OPT1	zt0	Zerotier Network	-

OPNsense Plugin Setup

1. Update OPNsense to the latest version System > Firmware > Updates
2. Install os-qemu-guest-agent and os-zerotier
3. Join Zerotier Network VPN > Zerotier > Join Network
4. Setup Network Interface or Import from previous backup

Jumphost VM

1. Update ansible/hosts with jumphost vm

ubuntu-server ansible_host=x.x.x.x ansible_user=devops

2. Run the install-server.yaml playbook with tags for each service to enable

Available Service

• Harbor - Container registry
• Minio - S3 Compatible storage
• Vault - Secret management
• NFS server for Kubernetes Cluster
• Zerotier - Remote access to Jumphost

ansible-playbook install-server.yaml --list-tags

playbook: install-server.yaml

  play #1 (ubuntu-server): Install Server       TAGS: []
      TASK TAGS: [always, harbor, minio, nfs, vault, zerotier]

Kubernetes Cluster

1. Provision Talos Kubernetes vm

terraform -chdir=./terraform plan

terraform -chdir=./terraform apply -target module.talos-k8s

2. Bootstrap Kubernetes Cluster, refer to talos directory for more details

3. Apply Kubernetes cluster essentials ( metrics-server / loadbalancer / nfs storage provisioner )

terraform -chdir=./terraform apply -target module.kubernetes

4. Update the VM network LoadBalancer IP in kubernetes/loadbalancer/metallb-address.yaml

kubectl apply -f ./kubernetes/loadbalancer

Authentication

Install Keycloak OIDC and PostgreSQL

kubectl apply -f ./kubernetes/db/postgres.yaml
kubectl apply -f ./kubernetes/keycloak/keycloak.yaml

Keycloak client for application available in ./kubernetes/keycloak/client

Addons

Apply base kubernetes addons

terraform -chdir=./terraform apply -target module.kubernetes-addons

Service Mesh and API Gateway

Install Mesh components and Patch Kong UI

kubectl apply -f ./kubernetes/monitoring/kiali.yaml 
kubectl apply -f ./kubernetes/monitoring/jaeger.yaml 

kubectl patch svc/kong-gateway-kong-manager --patch-file=./kubernetes/kong/kong-patch.yaml

Monitoring

Install Elastic and Kibana

kubectl apply -f ./kubernetes/elk/elastic.yaml
kubectl apply -f ./kubernetes/elk/kibana.yaml

Terraform Diagram

Generated using rover