Single-node Kubernetes cluster running Talos Linux on GCP, managed with Terraform and Nix.
The goal of this project is confidence. I want to be able to:
- Take it down anytime to save costs, knowing I can bring it back
- Redeploy from scratch without fear, because everything is codified
- Extend over the coming years, adding services without fragile manual steps
Everything is infrastructure as code. No SSH, no manual configuration, no "I forgot how I set that up." If it's not in this repo, it doesn't exist.
This repository provisions a minimal, immutable Kubernetes cluster:
- Talos Linux: Immutable, API-driven OS purpose-built for Kubernetes. No SSH, no shell, no package manager. Managed entirely via
talosctl. - Single control plane node: One
e2-medium(2 vCPU, 4GB RAM) innorthamerica-northeast1-a(Montreal). - Persistent storage: 20GB SSD attached for stateful workloads (PostgreSQL).
- Nix flake: Reproducible development environment with all required tools.
infrastructure/
├── .sops.yaml # SOPS config with age public key
├── flake.nix # Nix devshell (talosctl, kubectl, terraform, sops, age)
├── flake.lock # Locked dependencies (generated by nix)
├── terraform/
│ ├── persistent/ # Long-lived resources (~$4/month)
│ │ ├── main.tf # Google provider
│ │ ├── variables.tf # Project ID, region, zone
│ │ ├── storage.tf # Bucket (cafe-coding-fleet-artifacts) & Data disk (talos-data)
│ │ └── outputs.tf # Disk name
│ └── compute/ # Ephemeral resources (~$46/month)
│ ├── main.tf # Google + Cloudflare + SOPS providers
│ ├── variables.tf # Machine type, etc.
│ ├── data.tf # References persistent disk
│ ├── instance.tf # GCE instance running Talos
│ ├── networking.tf # Firewall rules (50000, 6443, 80, 443)
│ ├── cloudflare.tf # DNS A records
│ ├── outputs.tf # External IP for talosctl/kubectl
│ └── secrets.enc.yaml # Encrypted Cloudflare credentials
├── talos/
│ ├── .gitkeep # Placeholder
│ ├── controlplane.yaml # Generated - Talos machine config (DO NOT COMMIT)
│ ├── worker.yaml # Generated - Not used in single-node setup
│ └── talosconfig # Generated - talosctl client config (DO NOT COMMIT)
├── apps/
│ ├── nginx-ingress/
│ │ └── deploy.yaml # Ingress controller manifest
│ └── test-app/
│ ├── deployment.yaml # Test nginx deployment
│ ├── service.yaml # ClusterIP service
│ └── ingress.yaml # Ingress for test.justinmcintyre.com
├── scripts/
│ ├── cluster-up.sh # Start GCP cluster
│ ├── cluster-down.sh # Stop GCP cluster (exports certs, saves costs)
│ ├── local-cluster.sh # Start/stop local Docker cluster
│ ├── deploy-apps.sh # Deploy all applications
│ ├── deploy-ingress.sh # Deploy nginx ingress controller
│ ├── deploy-cert-manager.sh # Deploy cert-manager
│ ├── deploy-postgres.sh # Deploy PostgreSQL
│ ├── deploy-twenty.sh # Deploy Twenty CRM
│ ├── deploy-test-apps.sh # Deploy test applications
│ ├── monitor-status.sh # Monitor cluster health and certs
│ ├── dashboard.sh # Open Talos dashboard
│ ├── db-connect.sh # Connect to PostgreSQL
│ ├── disk-usage.sh # Show disk usage on node
│ ├── list-resources.sh # List all GCP resources
│ └── internal/ # Called by other scripts
│ ├── bootstrap-gcp.sh # Called by cluster-up.sh
│ ├── setup-gcp-image.sh # Called by cluster-up.sh
│ ├── test-local.sh # Called by local-cluster.sh
│ ├── export-certs.sh # Called by cluster-down.sh
│ └── require-env.sh # Ensures scripts run in nix shell
├── docs/
│ ├── kubernetes-basics.md # Pods, nodes, taints explained for single-node setup
│ ├── gcp-costs.md # Cost breakdown with pricing sources
│ ├── sops-secrets.md # SOPS/age key management guide
│ └── cloudflare-setup.md # API token + zone ID instructions
└── README.md # This file
-
Nix with flakes enabled
# If not installed, see https://nixos.org/download # Enable flakes in ~/.config/nix/nix.conf: experimental-features = nix-command flakes
-
Docker (for local testing only)
# Install Docker (distro-specific, see https://docs.docker.com/engine/install/) # Add yourself to docker group so sudo can access it sudo usermod -aG docker $USER # Log out and back in for group change to take effect
All other tools (talosctl, kubectl, terraform, gcloud, sops, age) are provided by the Nix flake.
All commands can be run via nix run without entering a shell:
# GCP cluster
nix run .#cluster-up
nix run .#cluster-down
# Local Docker cluster
nix run .#local-cluster -- up
nix run .#local-cluster -- down
# Deploy applications
nix run .#deploy-apps
# Utilities
nix run .#dashboard
nix run .#db-connect
nix run .#list-resourcesFor interactive work, enter the dev shell:
nix developThis gives you: talosctl, kubectl, terraform, gcloud, sops, age
nix develop -c gcloud auth application-default loginFor testing without GCP costs:
nix run .#local-cluster -- up
# Verify (inside nix develop)
nix develop
talosctl --nodes 10.5.0.2 health
kubectl get nodes
# Tear down
nix run .#local-cluster -- down# Create cluster (first run creates persistent disk, bucket, and Talos image)
nix run .#cluster-up
# Verify (inside nix develop)
nix develop
kubectl get nodes
kubectl get pods -AThe bootstrap script:
- Gets the node IP from terraform output
- Generates Talos machine config in
talos/ - Applies config to the node
- Bootstraps etcd
- Waits for cluster health
- Merges kubeconfig into
~/.kube/config
Terraform is split into two projects for cost savings (see "Shut Down to Save Costs" below).
| Resource | Name | Purpose |
|---|---|---|
google_compute_disk |
talos-data |
20GB SSD for PostgreSQL data |
google_storage_bucket |
{project}-artifacts |
General artifacts (future use) |
| Resource | Name | Purpose |
|---|---|---|
google_compute_instance |
talos-controlplane |
The Kubernetes node |
google_compute_firewall |
talos-api |
Port 50000 - talosctl access |
google_compute_firewall |
kubernetes-api |
Port 6443 - kubectl access |
google_compute_firewall |
http-https |
Ports 80/443 - web traffic |
cloudflare_record |
test, test2, crm |
DNS A records |
| Port | Protocol | Purpose |
|---|---|---|
| 50000 | TCP | Talos API (talosctl commands) |
| 6443 | TCP | Kubernetes API (kubectl commands) |
| 80 | TCP | HTTP (for ingress) |
| 443 | TCP | HTTPS (for ingress) |
If you've made changes to app manifests and want to redeploy without full bootstrap:
# Redeploy everything
./scripts/deploy-apps.sh
# Or redeploy individual components
./scripts/deploy-ingress.sh
./scripts/deploy-cert-manager.sh
./scripts/deploy-postgres.sh
./scripts/deploy-twenty.sh
./scripts/deploy-test-apps.shAll deploy scripts are idempotent - running them on an already-deployed app applies any changes.
# Talos-level health (etcd, kubelet, etc.)
talosctl --nodes <IP> --talosconfig talos/talosconfig health
# Kubernetes-level health
kubectl get nodes
kubectl get pods -A# Kernel messages
talosctl --nodes <IP> --talosconfig talos/talosconfig dmesg
# Service logs
talosctl --nodes <IP> --talosconfig talos/talosconfig logs kubelet
talosctl --nodes <IP> --talosconfig talos/talosconfig logs etcd# Check current version
talosctl --nodes <IP> --talosconfig talos/talosconfig version
# Upgrade (replace with actual version)
talosctl --nodes <IP> --talosconfig talos/talosconfig upgrade \
--image ghcr.io/siderolabs/installer:v1.7.0Terraform is split into persistent/ and compute/ projects specifically for this use case. When not using the cluster, destroy compute resources to pay only for disk storage:
| State | Monthly Cost |
|---|---|
| Running 24/7 | ~$70 CAD |
| Running 8hrs/day weekdays | ~$20 CAD |
| Shut down | ~$4 CAD (disk only) |
Part-time math: Compute is $66 CAD for 730 hrs ($0.09/hr). 8 hrs × 5 days × 4.33 weeks = 173 hrs × $0.09 = ~$16 + $4 disk = ~$20 CAD.
# Destroy VM, firewall, DNS (keeps persistent disk)
./scripts/cluster-down.sh
# Bring it back up
./scripts/cluster-up.shExample crontab (up at 8am, down at 6pm weekdays):
0 8 * * 1-5 cd /home/justinm/infrastructure && ./scripts/cluster-up.sh
0 18 * * 1-5 cd /home/justinm/infrastructure && ./scripts/cluster-down.shNote: You get a new IP each time. The bootstrap script handles this.
cd terraform
terraform destroyWarning: This deletes the VM AND the persistent disk. Data is lost.
GCP requires VM images to be in your project. The setup-gcp-image.sh script:
- Creates a temporary GCS bucket
- Downloads the Talos image from factory.talos.dev (~900MB)
- Uploads it to the temporary bucket
- Creates a GCP Compute Image from the upload
- Deletes the temporary bucket
This is called automatically by cluster-up.sh if the image doesn't exist. To upgrade Talos, update the version in the script and delete the old image.
The image uses the default Talos schematic (vanilla, no extensions). For custom images with extensions, generate a schematic ID at https://factory.talos.dev/ and update the script.
The data SSD (talos-data, 20GB by default) is attached to the instance at /dev/sdb (device name: data).
It is not formatted or mounted by default. This happens when PostgreSQL is deployed (see milestone 3).
To verify the disk is attached:
talosctl --nodes <IP> --talosconfig talos/talosconfig get disksThese contain secrets and are in .gitignore:
talos/*.yaml- Machine configs with cluster secretstalos/talosconfig- Client credentialsterraform/*.tfstate- May contain sensitive outputs*.agekey- SOPS encryption keys
The node hasn't finished booting or config wasn't applied:
# Check if node is reachable
ping <IP>
# Re-apply config
talosctl apply-config --insecure --nodes <IP> --file talos/controlplane.yamlUsually means etcd didn't start properly:
# Check etcd logs
talosctl --nodes <IP> --talosconfig talos/talosconfig logs etcd
# If stuck, you may need to reset and re-apply config
talosctl --nodes <IP> --talosconfig talos/talosconfig reset --graceful=false
talosctl apply-config --insecure --nodes <IP> --file talos/controlplane.yaml
talosctl bootstrap --nodes <IP> --endpoints <IP> --talosconfig talos/talosconfigThe Kubernetes API isn't ready yet:
# Check if API server is running
talosctl --nodes <IP> --talosconfig talos/talosconfig services
# Wait for health check to pass
talosctl --nodes <IP> --talosconfig talos/talosconfig healthEnsure Docker is running and you have permission:
# Check Docker is running and accessible
docker info
# If permission denied, add yourself to docker group
sudo usermod -aG docker $USER
# Then log out and back in
# If Docker not found, install it first
# See https://docs.docker.com/engine/install/Secrets are encrypted with SOPS using age keys. The encrypted file (terraform/secrets.enc.yaml) is safe to commit.
-
Generate an age key:
mkdir -p ~/.config/sops/age age-keygen -o ~/.config/sops/age/keys.txt
-
Copy the public key (starts with
age1...) and add it to.sops.yaml -
Create and encrypt secrets:
sops terraform/secrets.enc.yaml
Add your Cloudflare credentials:
cloudflare_api_token: "your-token" cloudflare_zone_id: "your-zone-id"
Copy your age private key to ~/.config/sops/age/keys.txt. Without this, terraform cannot decrypt secrets.
See docs/sops-secrets.md for the full guide.
- docs/kubernetes-basics.md - Pods, nodes, taints, and why we remove the control-plane taint
- docs/gcp-costs.md - Cost breakdown, pricing sources, and how to verify billing
- docs/sops-secrets.md - SOPS/age key management and troubleshooting
- docs/cloudflare-setup.md - Getting Cloudflare API token and zone ID
- Single node: Simplicity over HA. This is a personal/small project.
- Talos Linux: Immutable, secure, minimal attack surface. No SSH simplifies security.
- Montreal region: Low latency from Eastern Canada, data residency in Canada.
- e2-medium: Cheapest viable size (2 vCPU needed for Kubernetes).
- Separate data disk: Survives instance recreation, can snapshot independently.
- Nix flake: Reproducible tooling, no "works on my machine" issues.
~$70/month CAD. See docs/gcp-costs.md for details and pricing sources.
-
soft-serve - Git server for code backup
- SSH-based, minimal overhead
- Backup in case GitHub is down
- SQLite storage on persistent disk
-
Filebrowser - Simple file management
- Web-based file browser
- Upload/download files
- For Hermes agent file access