Skip to content

Latest commit

 

History

History
154 lines (106 loc) · 7.63 KB

runner-overview.adoc

File metadata and controls

154 lines (106 loc) · 7.63 KB
Raw
contentTags
platform
Cloud
Server v4.x
Server v3.x

CircleCI’s self-hosted runner overview

Use CircleCI’s self-hosted runner to run jobs on your own infrastructure. With runner, you can build and test on a wide variety of architectures, and gain additional control over the build environment.

Introduction

Self-hosted runner installation options are as follows:

  • Install in a Kubernetes cluster, using a container runner

  • Install in a machine execution environment using a machine runner

The diagrams below illustrate how CircleCI’s container and machine runners extend our existing systems.

CircleCI’s container runner architecture
Figure 1. Container runner architecture
CircleCI’s machine runner architecture
Figure 2. Machine runner architecture

Launch agent 1.1 deprecated

Launch agent 1.1 has been deprecated. Machine Runner 3.0 is the recommended replacement. Launch agent 1.1 will cease to be supported as of July 31, 2024.

CircleCI’s self-hosted runner operation

Once a self-hosted runner is installed, the runner polls circleci.com for work, runs jobs, and returns status, logs, and artifacts to CircleCI.

CircleCI runner use cases

CircleCI self-hosted runner is designed to meet the needs of two main use cases, as follows:

  • Privileged access and controls: Some customers need to be able to run jobs on-premises or on limited-access infrastructure due to strict isolation requirements. Self-hosted runner enables the following:

    • IP restrictions. Self-hosted runners can have static IP addresses that you can control.

    • Identity Access Management (IAM) permissions. If you set up self-hosted runners in AWS, they can be assigned IAM permissions.

    • Ability to monitor the operating system.

    • Ability to connect to private networks.

  • Unique compute requirements: Use self-hosted runners to run your jobs in an environment or architecture that CircleCI does not offer as a resource class.

Choosing a runner execution environment

CircleCI offers two types of self-hosted runners: container and machine.

Container runner

Container runner is installed in a Kubernetes cluster. Using Kubernetes enables you to run containerized jobs on self-hosted compute, similar to how jobs use the native Docker executor to run on CircleCI’s cloud platform. Container runner allows you to run hundreds of jobs at once, scaling pods effectively to meet compute demands. Container runner is a complement to the machine runner, not a replacement.

Note

CircleCI’s self-hosted runner has historically executed each job using a one-to-one mapping between the CI job and a machine environment (virtual or physical). Each machine runner would have the self-hosted runner binary installed on it. Running jobs in this manner sacrifices several benefits of a container-based solution that are afforded on CircleCI’s cloud platform when using the Docker executor:

  • The ability to seamlessly use custom Docker images during job execution.

  • Access to a consistent, clean, containerized build environment with every job.

After installation of the container-agent, the container runner will claim your containerized jobs, schedule them within an ephemeral pod, and execute the work within a container-based execution environment.

Container runner allows you to use CircleCI’s convenience images as well as custom Docker images.

Refer to the Container runner performance benchmarks page for a look at container runner performance.

Machine runner

Machine runner is installed either in a virtual machine, or natively, on a physical machine. Each machine runner job executes in the same environment (virtual or physical) where the self-hosted runner binary is installed. CircleCI’s machine runner can be installed on Linux, Windows, or macOS. Machine runner should be used if you are not running containerized CI jobs. Visit the Docker to machine page for more examples on when to use a machine execution environment.

If you do not use Kubernetes but still want to run your CI job in a container on a self-hosted runner, you can install the machine runner in Docker.

Machine runner is not compatible with CircleCI’s convenience images or custom Docker images.

Getting started

To get started with CircleCI’s self-hosted runners:

  • Provide your own platform for your self-hosted runners (see the following Available self-hosted runner platforms section for supported platforms).

  • You will need at least one credit on your account to use runners. Runner execution itself does not require credits but one credit is required in case your jobs use storage or networking. For more information, see the Persisting data overview.

  • For container runner installation, visit the Container runner installation page.

  • For machine runner, visit the installation guide for your desired platform:

Available self-hosted runner platforms

Supported level platforms ensure that CircleCI’s self-hosted runners are both built and tested on their respective systems.

Using a supported platform, you get the following:

  • Documentation and best practices.

  • Support. CircleCI customer engineers will assist you to resolve issues within the usual Advanced Service Level Agreements (SLAs)

Supported self-hosted runners are available on the following platforms:

Container and machine self-hosted runners:

  • Ubuntu 18.04 or later (x86_64, ARM64)

Container runners:

  • Kubernetes (x86_64, ARM64)

Machine runners:

  • macOS X 11.2+ (Intel, Apple M1)

  • Windows Server 2019, 2016 (x86_64)

  • Linux distributions - RHEL8, SUSE, Debian, etc (x86_64, ARM64, s390x, ppc64le)

CircleCI sometimes offers a preview level platform when a new platform for self-hosted runner is in active development. If there is a platform in a preview level, this section will be updated with information and limitations for that platform.

Limitations

Almost all standard CircleCI features are available for use with self-hosted runner jobs, however, a few features are not yet supported.

  • The following built-in environment variables are not populated within runner executors:

    • CIRCLE_PREVIOUS_BUILD_NUM

    • All deprecated cloud environment variables

  • Docker layer caching

  • For limitations relating to container runner, visit the Container runner page.