Skip to content

Serubin/flux-infra

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

96 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Personal k8s Cluster using Flux V2

Kubernetes clusters using the GitOps tool Flux. This git repository defines the configurations and apps that make up Kubernetes clusters. Flux SOPS integration is used to encrypt secrets with gpg.

Repository structure

The Git repository contains the following directories:

πŸ“
β”œβ”€πŸ“ apps            # kustomization and overlays for app installations per cluster
β”‚  β”œβ”€πŸ“ base         # apps available for installation
β”‚  β”œβ”€πŸ“ staging
β”‚  β””β”€πŸ“ prod
β”œβ”€πŸ“ charts          # helm chart repos
β”œβ”€πŸ“ clusters        # flux & gitops operator per cluster
β”‚  β”œβ”€πŸ“ staging
β”‚  β””β”€πŸ“ prod
β”œβ”€πŸ“ configs         # configs per cluster
β””β”€πŸ“ infrastructure
   β”œβ”€πŸ“ backup       # backup configurations (not operational)
   β”œβ”€πŸ“ crds         # cluster crds
   β””β”€πŸ“ ingress      # traefik ingress definitions

Software

The following apps are installed on the clusters.

Software Purpose
Flux2 GitOps Tool managing the cluster
Weave GitOps Powerful WebUI extension to Flux for deployment insights
Traefik Ingress Controller Cluster Ingress controller
Kube-Prometheus Stack Prometheus & Exporters to monitor the cluster
Grafana Monitoring & Logging Dashboard
Alertmanager Monitoring Alerts
Grafana Loki Log aggregation system
Authelia SSO & 2FA authentication server for Cluster Web Apps
Kutt URL Shortener
Vaultwarden Password Manager
Plausible Analytics
Tandoor-Receipes Receipe Manager and Meal Planner
Heimdall Static dashboard for the cluster applications
Lemonhope A simple discord bot

Automation & Service deployment

The Renovate Bot will automatically create prs to update services. Certain services have auto merging enabled, allowing the upstream repo to push tags, which in turn will automatically deploy to cluster.

Auto Deployment

To enable auto deployment on upstream repos, this build workflow can be used. It creates a calver tag for every push to the main/default branch. This will be picked up by renovating and auto-deploying (usually, this functionality is a little buggy).

.github/workflows/build.yaml name: Build and Publish Docker Image on: push: branches: ["main"]
env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

jobs:
  build-and-push-image:
    runs-on: ubuntu-latest

    permissions:
      contents: write
      packages: write

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4
        with:
          submodules: "true"
      - name: Log in to the Container registry
        uses: docker/login-action@v3.0.0
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Get next version
        uses: reecetech/version-increment@2023.9.3
        id: version
        with:
          scheme: calver
          increment: patch

      - name: Extract metadata to create tags and labels
        id: meta
        uses: docker/metadata-action@v5.5.1
        with:
          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
          tags: |
            type=raw,value=${{steps.version.outputs.version}},priority=300
            type=raw,value=latest,enable={{is_default_branch}}

      - name: Build and push Docker image
        uses: docker/build-push-action@v5.1.0
        with:
          context: .
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}

      - name: Create tag
        uses: actions/github-script@v5
        with:
          script: |
            github.rest.git.createRef({
              owner: context.repo.owner,
              repo: context.repo.repo,
              ref: 'refs/tags/${{ steps.version.outputs.version }}',
              sha: context.sha
            })

Todo

  • Custom charts / update some third party charts
  • Statping/Healthchecks
  • Plausible Analytics
  • Headscale VPN

Bootstrapping

  1. Install k3s on the server with
k3sup install --ip ${SSH_IP_ADDRESS} --local-path ${KUBE_CONFIG} --ssh-key "~/.ssh/${SSH_KEY}" --k3s-extra-args '--disable traefik'
  1. Add the flux-system namespace with
kubectl create namespace flux-system
  1. Add the decryption key k3s with
gpg --export-secret-keys --armor ${GPG_FINGERPRINT} | kubectl create secret generic sops-gpg --namespace=flux-system --from-file=sops.asc=/dev/stdin
  1. Bootstrap the cluster with
flux bootstrap github \
    --owner=serubin \
    --repository=flux-infra \
    --branch=main \
    --personal \
    --path=clusters/<cluster>

Environments

Staging Cluster

The staging cluster is a single-node Kubernetes instance (k3s) running on a Dell R410 on Debian.

Production Cluster

The production cluster is a 6c/12t 32GB RAM server running on a dedibox cloud.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published