Skip to content

aaronwittchen/gitlab-vm

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

70 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

GitLab VM + Debian Runners - OpenTofu/Terraform Proxmox

Lines of Code License GitHub Last Commit Quality Gate Status Maintainability Rating Reliability Rating Security Rating Vulnerabilities Code Smells Duplicated Lines (%) Bugs CodeFactor Codacy Badge

Arch Debian NixOS

ShellCheck JSON YAML XML Markdown KICS Grype Semgrep Trivy Checkov

Pre-commit Terraform Fmt TFLint Terraform Docs

OpenTofu/Terraform configuration to provision:

  • GitLab CE on a Debian 12 VM with optional Cloudflare Tunnel or local HTTPS via Caddy
  • GitLab Runners on Debian 12 VMs with K3s and Kubernetes executor
  • SonarQube on a Debian 12 VM for code quality analysis

Architecture

┌────────────────────────────────────────────────────────────────────────────────────────┐
│                                      Proxmox VE                                        │
│  ┌─────────────────────┐  ┌───────────────────┐  ┌───────────────┐  ┌───────────────┐  │
│  │   VM 1000 (Debian)  │  │  VM 1002 (Debian) │  │ VM 1003       │  │ VM 1004       │  │
│  │      GitLab CE      │  │  gitlab-runner-1  │  │ gitlab-runner-2│ │   SonarQube   │  │
│  │   192.168.68.50     │  │  K3s + Runner     │  │ K3s + Runner  │  │ 192.168.68.54 │  │
│  │                     │  │  192.168.68.52    │  │ 192.168.68.53 │  │               │  │
│  └─────────────────────┘  └───────────────────┘  └───────────────┘  └───────────────┘  │
└────────────────────────────────────────────────────────────────────────────────────────┘

Terraform Resource Graph

Terraform Graph

Prerequisites

  • Proxmox VE 7.x or 8.x
  • OpenTofu >= 1.6 (or Terraform >= 1.0)
  • Ansible >= 2.10 (for GitLab installation)
  • SSH key pair generated locally
  • Network access to Proxmox API
  • (Optional) Cloudflare account with a domain for HTTPS access

Install OpenTofu (Arch Linux)

sudo pacman -S opentofu

VM Specifications

VM OS CPU RAM Disk Purpose
1000 Debian 12 4 cores 16 GB 80 GB GitLab CE
1002 Debian 12 4 cores 16 GB 80 GB Runner 1
1003 Debian 12 4 cores 16 GB 80 GB Runner 2
1004 Debian 12 2 cores 8 GB 50 GB SonarQube

Quick Start

# 1. Configure variables
cp terraform.tfvars.example terraform.tfvars
# Edit terraform.tfvars with your values

# 2. Deploy infrastructure
tofu init
tofu apply

# 3. Install GitLab via Ansible
cd ansible
ansible-playbook playbook.yml

# 4. Create runner tokens in GitLab UI, then register runners
# GitLab → Admin → CI/CD → Runners → New instance runner
# SSH to each runner and register with Kubernetes executor (see below)

Setup

1. Create Debian Cloud Template on Proxmox

SSH into your Proxmox host and run:

cd /var/lib/vz/template/iso/
wget https://cloud.debian.org/images/cloud/bookworm/latest/debian-12-genericcloud-amd64.qcow2

qm create 9000 --name "debian-12-cloud" --memory 2048 --cores 2 --net0 virtio,bridge=vmbr0
qm importdisk 9000 debian-12-genericcloud-amd64.qcow2 local-lvm
qm set 9000 --scsihw virtio-scsi-pci --scsi0 local-lvm:vm-9000-disk-0
qm set 9000 --ide2 local-lvm:cloudinit
qm set 9000 --boot c --bootdisk scsi0
qm set 9000 --agent enabled=1
qm set 9000 --serial0 socket --vga serial0
qm template 9000

Enable snippets storage:

pvesm set local --content vztmpl,iso,snippets

2. Configure Terraform Variables

cp terraform.tfvars.example terraform.tfvars

Edit terraform.tfvars with your values:

Variable Description Example
proxmox_url Proxmox API URL https://192.168.68.2:8006
proxmox_password Proxmox root password
proxmox_node Proxmox node name pve
vm_ip Static IP for GitLab VM (CIDR) 192.168.68.50/24
gateway Network gateway 192.168.68.1

3. Deploy Infrastructure

tofu init
tofu plan
tofu apply

This creates:

  • GitLab VM (requires Ansible to install GitLab)
  • 2 GitLab Runner VMs with K3s and gitlab-runner pre-installed

4. Install GitLab with Ansible

Configure Ansible secrets:

cp ansible/group_vars/all/secrets.yml.example ansible/group_vars/all/secrets.yml

Run the playbook:

cd ansible
ansible-playbook playbook.yml

Alternative: Manual Installation

ssh admin@192.168.68.50
curl -fsSL https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | sudo bash
sudo EXTERNAL_URL="https://gitlab.local" apt-get install gitlab-ce

Get the initial root password:

sudo cat /etc/gitlab/initial_root_password

5. Register GitLab Runners

  1. Go to GitLab AdminCI/CDRunners
  2. Click New instance runner (create one for each runner)
  3. Copy the registration token

Register each runner with the Kubernetes executor:

# Runner 1
ssh admin@192.168.68.52

# Add GitLab hostname
echo '192.168.68.50 gitlab.local' | sudo tee -a /etc/hosts

# Add GitLab cert to trust store (for self-signed certs)
echo | openssl s_client -connect gitlab.local:443 2>/dev/null | openssl x509 | sudo tee /usr/local/share/ca-certificates/gitlab.crt
sudo update-ca-certificates

# Register with Kubernetes executor
export CI_SERVER_URL=https://gitlab.local
export RUNNER_TOKEN='<TOKEN_1>'
sudo -E gitlab-runner register --non-interactive \
  --url "$CI_SERVER_URL" \
  --token "$RUNNER_TOKEN" \
  --executor kubernetes \
  --kubernetes-namespace gitlab-runner \
  --kubernetes-image alpine:latest

# Runner 2
ssh admin@192.168.68.53
# Same steps with <TOKEN_2>

CI jobs will run as pods in the K3s cluster on each runner VM.

Runner Configuration

The runners are configured via variables.tf:

variable "runners" {
  default = {
    "gitlab-runner-1" = {
      vm_id = 1002
      ip    = "192.168.68.52/24"
    }
    "gitlab-runner-2" = {
      vm_id = 1003
      ip    = "192.168.68.53/24"
    }
  }
}

To add more runners, simply add entries to this map and run tofu apply.

Container Registry (Docker Hub)

Docker Hub is used for storing container images. Configure these CI/CD variables in GitLab:

Variable Value Protected Masked
DOCKERHUB_USERNAME Your Docker Hub username
DOCKERHUB_TOKEN Docker Hub access token

To create a Docker Hub access token:

  1. Go to Docker Hub → Account Settings → Security
  2. Click "New Access Token"
  3. Select "Read & Write" permissions

CI/CD

Example .gitlab-ci.yml that uses both runners:

stages:
  - test
  - build
  - push

test:
  stage: test
  image: alpine:latest
  script:
    - echo "Running tests..."

build:
  stage: build
  image: docker:24
  services:
    - docker:24-dind
  variables:
    DOCKER_TLS_CERTDIR: "/certs"
  script:
    - docker build -t $DOCKERHUB_USERNAME/$CI_PROJECT_NAME:$CI_COMMIT_SHA .
    - docker tag $DOCKERHUB_USERNAME/$CI_PROJECT_NAME:$CI_COMMIT_SHA $DOCKERHUB_USERNAME/$CI_PROJECT_NAME:latest

push:
  stage: push
  image: docker:24
  services:
    - docker:24-dind
  variables:
    DOCKER_TLS_CERTDIR: "/certs"
  script:
    - echo "$DOCKERHUB_TOKEN" | docker login -u "$DOCKERHUB_USERNAME" --password-stdin
    - docker push $DOCKERHUB_USERNAME/$CI_PROJECT_NAME:$CI_COMMIT_SHA
    - docker push $DOCKERHUB_USERNAME/$CI_PROJECT_NAME:latest
  only:
    - main

Jobs without tags will be distributed across available runners.

Local HTTPS with mkcert and Caddy

For local development with trusted HTTPS:

1. Generate Certificates

mkcert -install
mkcert gitlab.local "*.local"

2. Configure Caddy on GitLab VM

ssh admin@192.168.68.50

# Install Caddy
sudo apt install -y caddy

# Copy certs and configure
sudo mkdir -p /etc/caddy/certs
# scp your certs to /etc/caddy/certs/

sudo tee /etc/caddy/Caddyfile > /dev/null <<'EOF'
gitlab.local {
    tls /etc/caddy/certs/gitlab.local+1.pem /etc/caddy/certs/gitlab.local+1-key.pem
    reverse_proxy localhost:8080
}
EOF

sudo systemctl restart caddy

3. Add DNS Entry

Add to your local DNS or /etc/hosts:

192.168.68.50 gitlab.local

Destruction

tofu destroy

Troubleshooting

SSH Permission Denied

  • SSH user is admin, not root
  • Verify your SSH key: cat ~/.ssh/id_ed25519.pub
  • Clear old host key: ssh-keygen -R 192.168.68.52

gitlab-runner not installed

Cloud-init may have failed. Install manually:

ssh admin@192.168.68.52
curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash
sudo apt-get install -y gitlab-runner

TLS Errors During Registration

If using self-signed certs:

  1. Copy CA cert to runner: /etc/gitlab-runner/certs/ca.crt
  2. Add hosts entry: echo '192.168.68.50 gitlab.local' | sudo tee -a /etc/hosts
  3. Use --tls-ca-file /etc/gitlab-runner/certs/ca.crt during registration

Runner Not Picking Up Jobs

  1. Check runner status: sudo gitlab-runner list
  2. Verify runner is online in GitLab UI
  3. Ensure "Run untagged jobs" is enabled

K3s Issues

Check K3s status:

sudo systemctl status k3s
sudo kubectl get nodes
sudo kubectl get pods -n gitlab-runner

View job pod logs:

sudo kubectl logs -n gitlab-runner -l gitlab-runner=true

Monitoring

All VMs have prometheus-node-exporter installed and exposed on port 9100 for metrics collection.

VM IP Node Exporter
GitLab 192.168.68.50 :9100 (GitLab bundled)
Runner 1 192.168.68.52 :9100
Runner 2 192.168.68.53 :9100

Prometheus Configuration

Add these targets to your Prometheus scrape config:

scrape_configs:
  - job_name: 'gitlab-vms'
    static_configs:
      - targets:
          - '192.168.68.50:9100'
          - '192.168.68.52:9100'
          - '192.168.68.53:9100'
        labels:
          env: 'gitlab'

Grafana Dashboard

Import the Node Exporter Full dashboard (ID: 1860) in Grafana:

  1. Go to Grafana → +Import dashboard
  2. Enter ID: 1860
  3. Select Prometheus data source
  4. Use the instance dropdown to filter by GitLab VMs

Manual Node Exporter Installation

If node_exporter isn't installed on a runner:

ssh admin@192.168.68.52
sudo apt-get update && sudo apt-get install -y prometheus-node-exporter
sudo systemctl enable --now prometheus-node-exporter

Note: GitLab VM uses the bundled node_exporter. To expose it externally:

echo "node_exporter['listen_address'] = '0.0.0.0:9100'" | sudo tee -a /etc/gitlab/gitlab.rb
sudo gitlab-ctl reconfigure

SonarQube

Self-hosted code quality and security analysis.

Deploy SonarQube VM

cd sonarqube
cp terraform.tfvars.example terraform.tfvars
# Edit terraform.tfvars with your values
tofu init
tofu apply

Access at http://192.168.68.54:9000 (default: admin/admin).

GitLab Integration

Add CI/CD variables in GitLab (Settings → CI/CD → Variables):

Variable Value Masked
SONAR_HOST_URL http://192.168.68.54:9000 No
SONAR_TOKEN Token from SonarQube Yes

Add sonar-project.properties to your repo:

sonar.projectKey=your-project
sonar.qualitygate.wait=true

Add to .gitlab-ci.yml:

sonarqube:
  stage: analysis
  image: sonarsource/sonar-scanner-cli:latest
  variables:
    SONAR_USER_HOME: "${CI_PROJECT_DIR}/.sonar"
    GIT_DEPTH: "0"
  cache:
    key: sonar-${CI_PROJECT_NAME}
    paths:
      - .sonar/cache
  script:
    - sonar-scanner

GitHub Mirroring

Mirror to GitHub only when quality gate passes. Add CI/CD variables:

Variable Value Masked
GITHUB_USER GitHub username No
GITHUB_TOKEN GitHub PAT with repo scope Yes
GITHUB_REPO Repository name No

Add to .gitlab-ci.yml:

push-to-github:
  stage: mirror
  image: alpine/git
  variables:
    GIT_STRATEGY: clone
    GIT_DEPTH: "0"
  script:
    - git remote add github https://${GITHUB_USER}:${GITHUB_TOKEN}@github.com/${GITHUB_USER}/${GITHUB_REPO}.git || true
    - git push github HEAD:main --force
  rules:
    - if: $CI_COMMIT_BRANCH == "main"

The pipeline will only push to GitHub if SonarQube analysis passes (due to sonar.qualitygate.wait=true).

Archived: Talos Runner

The previous Talos Kubernetes cluster setup has been archived in archive/talos-runner/. The current setup uses K3s on Debian VMs for a simpler, lighter-weight Kubernetes executor.

Terraform Documentation

Requirements

Name Version
terraform >= 1.6.0
proxmox ~> 0.66

Providers

Name Version
proxmox ~> 0.66

Modules

No modules.

Resources

Name Type
proxmox_virtual_environment_file.cloud_init resource
proxmox_virtual_environment_file.runner_cloud_init resource
proxmox_virtual_environment_vm.gitlab resource
proxmox_virtual_environment_vm.runner resource

Inputs

Name Description Type Default Required
gateway Network gateway string n/a yes
proxmox_node Proxmox node name string n/a yes
proxmox_password Proxmox password string n/a yes
proxmox_url Proxmox API URL string n/a yes
proxmox_user Proxmox username string n/a yes
runner_cpu_cores CPU cores for runner VMs number 4 no
runner_disk_size Disk size in GB for runner VMs number 80 no
runner_memory Memory in MB for runner VMs number 16384 no
runners Map of GitLab runners to create
map(object({
vm_id = number
ip = string
}))
{
"gitlab-runner-1": {
"ip": "192.168.2.52/24",
"vm_id": 1002
},
"gitlab-runner-2": {
"ip": "192.168.2.53/24",
"vm_id": 1003
}
}
no
snippets_storage Storage for cloud-init snippets and ISOs string "local" no
ssh_public_key_file Path to SSH public key file string "~/.ssh/id_ed25519.pub" no
storage Storage for VM disk string "local-lvm" no
template_id VM template ID to clone from number 9000 no
vm_id VM ID for GitLab number 1000 no
vm_ip Static IP for GitLab VM (CIDR notation) string n/a yes

Outputs

Name Description
runner_ips IP addresses of GitLab runners
vm_ip IP address of the GitLab VM

References

About

No description, website, or topics provided.

Resources

License

Contributing

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages