Move to Kubernetes

[Xe]

TODO fixes

• Fix Patreon API calls
• Mi is too aggressive, limit to hostname xeiaso.net
• Document that I'm stashing all the data in /data/xesite on the worker nodes directly, this is a bad idea
• Figure out kustomize variables to annotate docker builds with commit hash
• The website pods do a site build on app start, probably need to cache these in Tigris and push them to Tigris in CI? -- probably not a problem due to the pod disruption budgets
• Write blogpost
• Rip out the zipfile
• CD

Big deploy

• Change REDIRECT_DOMAIN to xeiaso.net when doing the big deploy
• Rename ZZ_MIMI_ANNOUNCE_URL in 1password to MIMI_ANNOUNCE_URL when mi is patched to only report for xeiaso.net

Things I learned for the blogpost

• Needed to configure RBAC to get patreon-saasproxy to properly store its state in k8s secrets
• Need to be careful about pushing this to main, that will break the existing partreon-saasproxy setup on fly.io if I am not careful
• Annoying that I can't change the CIDRs for vultr k8s
• Wish I could change the cluster DNS name for vultr k8s to rhadamanthus.xeserv.us so I can DNS/IP route it and access services like they were local like I have in my homelab, 4via6?
• Wish I could beanstalk-interconnect alrest and rhadamanthus
• Posting to mastodon made the logs unreadable lol
• PodDisruptionBudgets are great, but you need a readinessProbe for them, readiness is different than healthiness
• Civo k8s w/talos was kinda cursed, had issues with TLS certificates and metrics-server, tried k3s instead
• Kinda annoying that with 3 clusters I have as many CNIs to deal with, but whatever
• k3s worked goddamn instantly lol
• Cannot use the same domain from multiple clusters with external-dns: Allow multiple A records for the same domain from different external-dns instances. kubernetes-sigs/external-dns#1441
• Set up a tor hidden service with the new deployment, was basically seamless, had to patch domain_redirect.go to ignore it
• Found a subtle bug with how domain_redirect.go would handle non-GET requests, fixed
• Destroying rhadamanthus, Civo cloud is the ultimate winner for this due to no egress fees
• adding the Onion-Location header was seamless
• https://github.com/bugfest/tor-controller
• Ripping out the zipfile means that link embeds on Slack and LinkedIn
• had to MachineProxy back to fly.io for getting patreon API access working, need to investigate later