Terraform modules for creating autoscaling groups of private GitHub Actions Runners in your AWS account.
This repo contains three different Terraform modules, each themed around launching autoscaling groups of GitHub Actions Runners in your own AWS account.
Deploys an AWS ImageBuilder pipeline and associated resources to create a docker container that is able to run Terraform commands. This container will be used to execute GitHub Actions jobs in your workflows.
You don't have to use this module if you already have your own container image that is able to run Terraform; this is just a barebones container that is minimally functional.
Deploys an AWS ImageBuilder pipeline and associated resources to create an EC2 AMI that features all of the prerequisites for registering a GitHub Actions runner.
This includes:
- Base Ubuntu 20.04 (Focal) image
- Baseline packages required from the OS package repository, e.g.
curl,git,cron, etc. - SSM Agent
- CloudWatch Logs Agent
- The GitHub Actions Runner agent binary, either downloaded from GitHub's own releases page, or from your own S3 cache
- Pre-cached terraform container image (optional)
Autoscaling EC2 instances will be launched with AMIs built from this pipeline.
Deploys an autoscaling group of EC2 instances that each register as a self-hosted runner with a GitHub project of your choice. The autoscaling is performed based on a custom CloudWatch metric generated by a Lambda that polls your GitHub project to determine how many jobs are running/pending.
The module also includes:
| Feature | Purpose |
|---|---|
| EC2 Launch Template | that will deploy instances into the VPC of your choosing |
| EC2 Autoscaling Group | with scaling rules controlled by a schedule and/or an accurate custom metric (both optional) |
| EC2 IAM Role & Instance Profile | to allow access to the AWS APIs required to deploy your application (you can customise this). Also allows SSM Session Manager access for when your runners are in Private Subnets |
| EC2 Security Group | to protect instances launched by the ASG and provide minimal outbound traffic |
| CloudWatch Log Groups | for key OS logs with configuration to stream those logs to CloudWatch from each EC2 instance in the ASG, using the CloudWatch Logs Agent |
| CloudWatch Metric Filter & Alarms | to notify about failed instances |
| CloudWatch Dashboard | to monitor the autoscaling activity and key EC2 metrics (CPU, Memory, IO) |
| Secrets Manager Secret | with a placeholder value, that should be overwritten with a valid Personal Access Token (PAT) for the GitHub project you wish to register with |
| Bootstrapping BASH code | to self-register the instance with a GitHub server, organisation and project of your choosing |
| Multi-Registration | gives you the ability to register a single EC2 instance as multiple independent GitHub Runner agents, so you can run parallel jobs on a single instance. This is good for saving on infrastructure costs |
| Lambda Function | that polls the GitHub API to determine how busy your project is (i.e. how many pending/running jobs) and writes custom CloudWatch Metrics based on the response |
| CloudWatch Event Rule | to run the Lambda function every minute during the working day (as defined by you) |
| Scale In Protection | implemented as a systemd service that calls the AWS API, this prevents a runner from being scaled-in whilst it is busy running a job |
| Automatic De-Registration | when your ASG scales in and instances are destroyed, those destroyed instances automatically de-register themselves from your GitHub project so they will receive no further jobs |
-
Start with the imagebuilder-terraform-container module, but only if you don't already have a suitable container for running jobs. Create a file in your terraform project that calls this module with appropriate parameters as outlined below. The result will be an imagebuilder pipeline that creates a docker container for running Terraform commands.
-
The pipeline is set to run on a schedule by default, but you should immediately run the pipeline to generate a baseline container for later use. This will take around 20 minutes to complete.
-
Now onto the imagebuilder-github-runner-ami module. You will always need this one. Create a file in your terraform project that calls this module with appropriate parameters as outlined below. The result will be an imagebuilder pipeline that creates an EC2 AMI with all of the pre-requisites for running the GitHub Agent.
-
The pipeline is set to run on a schedule by default, but you should immediately run the pipeline to generate an AMI for later use. This will take around 45 minutes to complete.
-
Finally onto the autoscaling-github-runners module. This is the module that creates the actual runners, so you will always need this one too. Create a file in your terraform project that calls this module with appropriate parameters as outlined below
-
Once fully deployed the ASG will begin deploying EC2 instances (as per your min/max/desired requirements) immediately; however, these instances will not connect to GitHub successfully because they don't have a valid access token yet. Navigate to Secrets Manager and find the placeholder secret that has been created for you. Its name will end in the word
pat(also check the description field to be sure). Overwrite the value of this secret with a valid Personal Access Token from the GitHub project you want to register with. -
Terminate all running EC2 instances and let them be replaced by the ASG. This time they should successfully register with your GitHub project and be visible in the page
Settings -> Actions --> Runners -
In your GitHub Actions Workflow YAML, ensure your jobs feature a
runs-onsection that includes the tagself-hostedand the name of the tag you gave your runners in theec2_github_runner_tag_listinput variable. This will ensure that your workflow jobs will be deployed to your private self-hosted runners inside your own AWS account.
runs-on:
- self-hosted
- techtestBy default the runners will only have basic permissions to write their own logs and metrics, pull containers from ECR, and read/write files to a CI/CD artifacts bucket defined by you.
If you want to give your runners greater permissions (and you probably do, because they need to have permission to do things like deploy an application) then you can supply additional IAM Policies in the input variable ec2_iam_role_extra_policy_attachments, which is a list of ARNs.