Skip to content

gitlab‐runner

Jump to bottom
KamilCuk edited this page Sep 24, 2025 · 5 revisions

How to register

Check https://docs.gitlab.com/runner/register/

  1. In gitlab web go to build->runners and click "Create runner"

  2. Get the gitlab-runner registering command.

  3. Register runner with custom executor.

  4. Edit /etc/gitlab-runner/config.toml configuration file to add the runners.custom section.

    ```
 # /etc/gitlab-runner/runner.toml
 [[runners]]
   id = 27898742
   executor = "custom"
   [runners.custom]
     config_exec = "nomadtools"
     config_args = ["gitlab-runner", "config"]
     prepare_exec = "nomadtools"
     prepare_args = ["gitlab-runner", "prepare"]
     run_exec = "nomadtools"
     run_args = ["gitlab-runner", "run"]
     cleanup_exec = "nomadtools"
     cleanup_args = ["gitlab-runner", "cleanup"]
 ```

  5. Create gitlabrunner Nomad namespace nomad namespace apply gitlabrunner

  6. Create Nomad policy that is able to execute nomad alloc exec and nomad acl policy apply gitlabrunner gitlabrunner.policy.hcl it.

    namespace "gitlabrunner" {
        # For creating jobs.
        policy = "write"
        # To allow 'raw_exec' to execute anything.
        capabilities = ["alloc-node-exec"]
    }
  1. Create nomad token with gitlabrunner policy nomad acl token create -policy=gitlabrunner.
  2. Create /etc/gitlab-runner/nomad-gitlab-runner.yml file with the following content:
---
default:
  # You can use any NOMAD_* variables here.
  NOMAD_TOKEN: "<token>"
  NOMAD_NAMESPACE: gitlabrunner
  CPU: 2048
  MemoryMB: 2048
  docker:
     services_privileged: true
  1. Confirm everything ok with nomadtools gitlab-runner showconfig.
  2. For firewall-cmd I needed to run firewall-cmd --zone docker --add-interface nomad`

firewall-cmd

I needed to run firewall-cmd --zone docker --add-interface nomad to add the nomad bridge interface to allow to access internet.

What is it

Allows running not the gitlab-runner itself in Nomad. Rather, the gitlab-runner is running on (any) one host. That gitlab-runner will schedule Nomad jobs to execute. These jobs will execute the CI/CD from Gitlab.

The gitlab-runner script runs a background Nomad job for the whole duration of Gitlab CI/CD task. There are 3 modes available that you can specify in configuration file: raw_exec, exec and docker mode.

In raw_exec and exec modes, the Nomad job has one task. It is not supported to specify services in gitlab-ci.yml. On each stage of gitlab-runner executor is executed with nomad alloc exec inside the task spawned in Nomad with a provided entrypoint script. This script adjusts niceness levels, adjusts OOM killer, sets taskset -s and switches user with runuser -u and runs bash shell if available with falling back to sh shell.

In docker mode, the Nomad job has multiple tasks, similar to gitlab-runner docker executor spawning multiple images. One task is used to clone the repository and manage artifacts, exactly like in https://docs.gitlab.com/runner/executors/docker.html#docker-executor-workflow in gitlab-runner docker executor. The other task is the main task of the job. It does not run the job image entrypoint. All commands are executed with nomad alloc exec with the custom entrypoint wrapper. In this case the wrapper does not use taskset nor runuser, as these parameters are set with docker configuration.

Specifying services in .gitlab-ci.yml in docker mode is supported. Each service is a separate task in the group in the Nomad job. The Nomad job runs the task group in bridge mode docker networking, so that all tasks share the same network stack. One additional "waiter" task is created that runs prestart to wait for the services to respond. This is meant to mimic the default gitlab-runner docker executor behavior described in https://docs.gitlab.com/runner/executors/docker.html#how-gitlab-runner-performs-the-services-health-check .

In order for services to work, it is hard coded that the waiter helper image starts with /var/run/docker.sock mounted inside to connect to docker. Additionally, Nomad has to support 'bridge' docker network driver for Nomad to start the job. See https://developer.hashicorp.com/nomad/docs/networking#bridge-networking .

Configuration

nomad-gitlab-runner uses configuration stored in /etc/gitlab-runner/nomad-gitlab-runner.yml file. Execute nomadtools gitlab-runner showconfig to print current program configuration. The example shortest sensible needed configuration file is presented below.

# /etc/gitlab-runner/nomad-gitlab-runner.yml
---
default:
  NOMAD_TOKEN: "1234567"
  CPU: 2048
  MemoryMB: 2048
  docker:
    services_privileged: true

27898742:
  mode: docker

This is another longer example of configuration file with some comments:

# /etc/gitlab-runner/nomad-gitlab-runner.yml
---
default:
  # You can use NOMAD_* variables here.
  NOMAD_TOKEN: "12345"
  NOMAD_NAMESPACE: gitlabrunner
27898742:
  mode: docker
  CPU: 2048
  MemoryMB: 2048
  MemoryMBMax: 2048
  docker:
    # Default docker image.
    image: "alpine:latest"
    # Set to true to be able to run dind service.
    services_privileged: true
  override:
    task_config:
      cpuset_cpus: "2-8"

The configuration file has a default section that applies to all configurations. Then, each key is the runner ID as specified in config.toml of gitlab-runner configuration. Each key has a separate configuration.

The Nomad job by default makes available the standard Nomad directories. The alloc directory is shared between all the tasks, including "waiter" task, cloning task and services specified in .gitlab-ci.yml.

To use docker:dind you can use the standard configuration if you use auto_fix_docker_dind configuration option. auto_fix_docker_dind is by default true and if DOCKER_TLS_CERTDIR is specified in the variables, then nomad-gitlab-runner mounts allocation at /certs and sets other needed DOCKER_* variables by itself.

# .gitlab-ci.yml
---
docker_dind_auto:
  image: docker:24.0.5
  services:
    - docker:24.0.5-dind
  variables:
    # When the configuration option auto_fix_docker_dind is set to true, then:
    DOCKER_TLS_CERTDIR: "/certs"
  script:
    - docker info
    - docker run -ti --rm alpine echo hello world

If auto_fix_docker_dind is false, you need to set DOCKER_* variables yourself just like for Kubernetes executor. Use /alloc directory to share the certificates between client and server.

# .gitlab-ci.yml
---
docker_dind_alloc:
  variables:
    DOCKER_CERT_PATH: "/alloc/client"
    DOCKER_HOST: tcp://docker:2376
    DOCKER_TLS_CERTDIR: "/alloc"
    DOCKER_TLS_VERIFY: 1
  script:
    - docker info
    - docker run -ti --rm alpine echo hello world

Clone this wiki locally