Autoscale GitLab Runner on Google VM

The purpose of this module is to create "GitLab Manager" VM, configure it and register "GitLab runner(s)" which allows automatically spin up and down VMs to make sure your builds get processed immediately.

Features

• Use preemptible or regular on-demand VMs
• There is ability to configure from 1 to N amounts of runners inside gitlab_register_runner variable each with its own attributes
• Supports docker images for CI/CD jobs stored in private GCR rregistry. Such code in .gitlab-ci.yml is available:

test:
  stage: mystage
  image: us.gcr.io/<project_id>/<path_to_gcr>/<image_name>:<number>
  script:
  ...

• Forked version of docker+machine executor
• Distributed runners caching is enabled from the box

Requirements

Runners can be configured as shared|group|specific. Based on the above the appropriate registration token should be taken. For more info please refer here

Example of module usage:

module "gitlab-runner" {
  source   = "allyrr/gitlab-runner/google"

  gcp_gitlab_resource_prefix = "gitlab"
  gcp_project_id = "YOUR_PROJECT"
  gitlab_registration_token = "YOUR_TOKEN"
  gcp_region = "us-central1"
  gcp_zone = "us-central1-a"

  gitlab_register_runner = {
    "runner1" = {
      "runner_limit"=10
      "runner_request_concurrency"=1
      "runner_url"="https://gitlab.com/"
      "runner_idle_nodes"=0
      "runner_idle_time"=300
      "runner_max_builds"=10
      "runner_machine_type"="n2-standard-2"
      "runner_preemptible"=true
      "runner_disk_size"=50
      "runner_description"="gitlab-runner1"
      "runner_tag"="gcp-n2-standard-8gb"
    }
    "runner2" = {
      "runner_limit"=3
      "runner_request_concurrency"=1
      "runner_url"="https://gitlab.com/"
      "runner_idle_nodes"=1
      "runner_idle_time"=0
      "runner_max_builds"=10
      "runner_machine_type"="n2-standard-2"
      "runner_preemptible"=false
      "runner_disk_size"=50
      "runner_description"="gitlab-runner2"
      "runner_tag"="gcp-n2-standard-8gb"
    }
  }
}
output "gitlab-manager" {
  value = module.gitlab-runner.gitlab-manager
}

Providers

Name	Type
google-beta_google_compute_address.main	resource
google_compute_firewall.allow-all-internal	resource
google_compute_firewall.allow-ssh	resource
google_compute_instance.gitlab_manager	resource
google_compute_network.main	resource
google_compute_router.main	resource
google_compute_router_nat.main	resource
google_compute_subnetwork.main_subnet_0	resource
google_project_iam_member.sa_gitlab	resource
google_service_account.sa_gitlab	resource
google_service_account_key.sa_gitlab	resource
google_storage_bucket.bucket_cache	resource
google_storage_bucket_iam_binding.gitlab_runner	resource
random_id.bucket_id	resource
random_id.instance_id	resource
google_compute_default_service_account.default	data source
template_file.stage1_config	data source
template_file.stage2_config	data source
template_file.stage3_config	data source

Inputs

Name	Description	Type	Default	Required
gcp_gcs_cache_age	Time in days to keep the GitLab's cache in the GCS bucket	number	60	no
gcp_gitlab_resource_prefix	Name prefix for all the resources	string	"gitlab"	no
gcp_main_vpc_sub_ip_range	GCP default subnetwork ip range	string	"10.10.0.0/20"	no
gcp_project_id	GCP project id	string	n/a	yes
gcp_region	GCP default region	string	n/a	yes
gcp_zone	GCP default zone	string	n/a	yes
gitlab_docker_machine_release	Release version of forked docker-machine. Available releases: https://gitlab.com/gitlab-org/ci-cd/docker-machine/-/releases	string	"v0.16.2-gitlab.11"	no
gitlab_register_runner	Map of different GitLab runners and their attributes. For more info refer 1) Google Machine drivers - https://docs.docker.com/machine/drivers/gce/ and 2) GitLab One-line registration command https://docs.gitlab.com/runner/register/#one-line-registration-command	any	n/a	yes
gitlab_registration_token	Registration token. Can be found under Settings > CI/CD and expand the Runners section of group you want to make the runner work for	string	n/a	yes

Outputs

Name	Description
gitlab-manager	String to SSH into a GitLab manager VM via IAP

Sponsors

This work has been sponsored by Scotts Miracle Gro and LITSLINK.