terraform-aws-github-runners

Simple to use, self-hosted GitHub Action runners. Uses EC2 spot instances with configurable AutoScaling.

---

Features

• Simple! See the provided examples for a quick-start.
• Cost-effective. Uses EC2 Spot pricing and AutoScaling to keep costs low. Runs multiple runners per EC2 instance depending on the number of vCPU available.
• Customisable using cloudinit.
• Scalable. By default one runner process and 20GB storage is provided per vCPU per EC2 instance.

Why?

Deploying a self-hosted github runner should be simple. It shouldn't need a long setup process or a lot of infrastructure.

This module additionally does not require public inbound traffic, and can be easily customised if needed.

Known limitations

1. Needs a VPC.

Currently this module requires a VPC and Subnets for deployment. In future a non-VPC deployment could perhaps be added.

2. Changes may affect the shared EC2 environment.

Parallel runners are ephemeral and their work environment is destroyed after each job is done. However, they still run on the same underlying EC2 instance. This means they can make changes which impact each other, for example if the EBS storage gets full.

A possible workaround could be to run jobs in a container.

How it works

An AutoScaling group is created to spin up Spot EC2 instances on a schedule. The instances retrieve a pre-configured GitHub access token from AWS SSM Parameter Store, and start one (or more) ephemeral actions runner processes. These authenticate with GitHub and wait for work.

Steps execute arbitrary commands, defined by your repo workflows.

For example:

• Perform a linting check.
• Connect to another AWS Account using an IAM credential and operate on some EC2 or RDS infrastructure.
• Anything else...

A full list of created resources is shown below.

How to use it

1. Store your GitHub token

Create a GitHub personal access token. Add it to AWS Systems Manager Parameter Store with the SecureString type.

2. Configure module

Configure and deploy the module using Terraform. See examples below.

More info

• Found an issue? Want to help? Contribute.
• Review a cost estimate.

Module Docs

Basic Example

module "github_runner" {
  source = "../../"

  # Required parameters
  ############################
  region     = "af-south-1"
  github_url = "https://github.com/my-org"

  # Naming for all created resources
  naming_prefix = "test-github-runner"

  ssm_parameter_name = "/github/runner/token"

  # 2 cores, so 2 ephemeral runners will start in parallel.
  ec2_instance_type = "t3.micro"

  vpc_id     = "vpc-0ffaabbcc1122"
  subnet_ids = ["subnet-0123", "subnet-0456"]
}

Advanced Example

locals {
  naming_prefix = "test-github-runner"
  vpc_id        = "vpc-0ffaabbcc1122"
}

# Create a custom security-group to allow SSH to all EC2 instances
resource "aws_security_group" "this" {
  name        = "${local.naming_prefix}-sg"
  description = "GitHub runner ${local.naming_prefix}-sg"

  # tfsec:ignore:aws-ec2-no-public-egress-sgr
  egress {
    description = "egress"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  vpc_id = local.vpc_id
  #checkov:skip=CKV2_AWS_5:The SG is attached by the module.
}

data "http" "myip" {
  url = "http://ipv4.icanhazip.com"
}

resource "aws_security_group_rule" "ssh_ingress" {
  description       = "Allow SSH ingress to EC2 instance"
  type              = "ingress"
  from_port         = 22
  to_port           = 22
  protocol          = "tcp"
  cidr_blocks       = ["${chomp(data.http.myip.body)}/32"]
  security_group_id = aws_security_group.this.id
}

module "github_runner" {
  source = "../../"

  # Required parameters
  ############################
  region     = "af-south-1"
  github_url = "https://github.com/my-org"

  naming_prefix = local.naming_prefix

  ssm_parameter_name = "/github/runner/token"

  ec2_instance_type = "t3.micro"

  vpc_id     = local.vpc_id
  subnet_ids = ["subnet-0123", "subnet-0456"]

  # Optional parameters
  ################################

  # If for some reason you dont want to install everything.
  software_packs = [
    "BASE_PACKAGES", # Extra utility packages like curl, zip, etc
    "docker-engine",
    "node",
    "python2" # Required for cloudwatch logging
  ]

  ec2_associate_public_ip_address = true
  ec2_key_pair_name               = "my_key_pair"
  security_groups                 = [aws_security_group.this.id]

  autoscaling_max_instance_lifetime = 86400
  autoscaling_min_size              = 2
  autoscaling_desired_size          = 2
  autoscaling_max_size              = 5

  autoscaling_schedule_time_zone = "Africa/Johannesburg"
  # Scale up to desired capacity during work hours
  autoscaling_schedule_on_recurrences = ["0 07 * * MON-FRI"]
  # Scale down to zero after hours
  autoscaling_schedule_off_recurrences = ["0 18 * * *"]

  cloud_init_extra_packages = ["neofetch"]
  cloud_init_extra_runcmds = [
    "echo \"hello world\" > ~/test_file"
  ]

  cloudwatch_log_group = "/some/log/group"
}

Software packs

locals {
  # All available software packs
  all = [
    # Contains base packages eg curl, zip, etc
    "BASE_PACKAGES",

    "docker-engine",
    "node",
    "pre-commit",
    "python2",
    "python3",
    "terraform",
    "terraform-docs",
    "tflint",
    "tfsec"
  ]
}

---

Inputs

Name	Description	Type	Default	Required
ami_name	AWS AMI name filter for launching instances. GitHub supports specific operating systems and architectures, including Ubuntu 22.04 amd64 which is the default. Note: The included software packs are not tested with other AMIs.	string	"ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-20220609"	no
ami_owners	AWS AMI owners to limit AMI search. Values may be an AWS Account ID, "self", or an AWS owner alias eg "amazon".	list(string)	[ "amazon" ]	no
autoscaling_desired_size	The number of Amazon EC2 instances that should be running. When scaling_mode="autoscaling-group"	number	1	no
autoscaling_max_instance_lifetime	The maximum amount of time, in seconds, that an instance can be in service. Values must be either equal to 0 or between 86400 and 31536000 seconds. When scaling_mode="autoscaling-group"	string	0	no
autoscaling_max_size	The maximum size of the Auto Scaling Group. When scaling_mode="autoscaling-group"	number	3	no
autoscaling_min_size	The minimum size of the Auto Scaling Group. When scaling_mode="autoscaling-group"	number	1	no
autoscaling_schedule_off_recurrences	A list of schedule cron expressions, specifying when the Auto Scaling Group will terminate all instances. Example: ["0 18 * * *"] When scaling_mode="autoscaling-group"	list(string)	[]	no
autoscaling_schedule_on_recurrences	A list of schedule cron expressions, specifying when the Auto Scaling Group will launch instances. Example: ["0 07 * * MON-FRI"] When scaling_mode="autoscaling-group"	list(string)	[]	no
autoscaling_schedule_time_zone	The timezone for schedule cron expressions. https://www.joda.org/joda-time/timezones.html When scaling_mode="autoscaling-group"	string	""	no
cloud_init_extra_other	Arbitrary text to append to the cloudinit script.	string	""	no
cloud_init_extra_packages	A list of strings to append beneath the packages: section of the cloudinit script. https://cloudinit.readthedocs.io/en/latest/topics/modules.html#package-update-upgrade-install	list(string)	[]	no
cloud_init_extra_runcmds	A list of strings to append beneath the runcmd: section of the cloudinit script. https://cloudinit.readthedocs.io/en/latest/topics/modules.html#runcmd	list(string)	[]	no
cloud_init_extra_write_files	A list of strings to append beneath the write_files: section of the cloudinit script. https://cloudinit.readthedocs.io/en/latest/topics/modules.html#write-files	list(string)	[]	no
cloudwatch_log_group	CloudWatch log group name prefix. Runner logs from /var/log/syslog are sent here. Example: github_runner, with this value logs will be written to github_runner/var/log/syslog/<instance_id>. If left unspecified then logging is disabled.	string	""	no
create_iam_resources	Should the module create the IAM resources needed. If set to false then an "iam_instance_profile_arn" must be provided.	bool	true	no
ec2_associate_public_ip_address	Whether to associate a public IP address with EC2 instances in a VPC.	bool	false	no
ec2_ebs_volume_size	Size in GB of instance-attached EBS storage. By default this is set to per_instance_runner_count * 20 GB.	number	-1	no
ec2_instance_type	Instance type for EC2 instances.	string	n/a	yes
ec2_key_pair_name	EC2 Key Pair name to allow SSH to EC2 instances.	string	""	no
github_organisation_name	GitHub orgnisation name. Derived from github_url by default.	string	""	no
github_runner_group	Custom GitHub runner group.	string	""	no
github_runner_labels	Custom GitHub runner labels. Example: "gpu,x64,linux".	list(string)	[]	no
github_url	GitHub organisation URL. Example: "https://github.com/cloudandthings/".	string	n/a	yes
iam_instance_profile_arn	IAM Instance Profile to launch EC2 instances with. Must allow permissions to read the SSM Parameter. Will be created by default.	string	""	no
iam_policy_arns	A list of existing IAM policy ARNs to attach to the runner IAM role.	list(string)	[]	no
naming_prefix	Created resources will be prefixed with this.	string	"github-runner"	no
per_instance_runner_count	Number of runners per instance. By default this is set to num_vCPUs * num_cores * threads_per_core. May be set to 0 to never create runners.	number	-1	no
region	AWS region.	string	n/a	yes
scaling_mode	How instances are managed. Can be either "autoscaling-group" or "single-instance".	string	"autoscaling-group"	no
security_groups	A list of security groups to assign to EC2 instances. Note: If none are provided, a new security group will be used which will deny inbound traffic including SSH.	list(string)	[]	no
software_packs	A list of pre-defined software packs to install. Valid options are: "ALL", "BASE_PACKAGES", "docker-engine", "node", "python2", "python3", "terraform", "terraform-docs", "tflint", "tfsec". An empty list will mean none are installed.	list(string)	[ "ALL" ]	no
ssm_parameter_name	SSM parameter name for the GitHub Runner token. Example: "/github/runner/token".	string	n/a	yes
subnet_ids	The list of Subnet IDs to launch EC2 instances in. If scaling_mode="single-instance" then the first Subnet ID from this list will be used.	list(string)	n/a	yes
vpc_id	The VPC ID to launch instances in.	string	n/a	yes

---

Modules

Name	Source	Version
software_packs	./modules/software	n/a
user_data	./modules/user_data	n/a

---

Outputs

Name	Description
aws_instance_id	Instance ID (when scaled_mode=single-instance)
aws_instance_public_ip	Instance public IP (when scaled_mode=single-instance)
per_instance_runner_count	Effective per instance runner count.
software_packs	List of software packs that were installed.

---

Providers

Name	Version
aws	~> 4.9
null	~> 3.2

---

Requirements

Name	Version
terraform	>= 0.14.0
aws	~> 4.9
http	~> 3.0
null	~> 3.2

---

Resources

Name	Type
aws_autoscaling_group.this	resource
aws_autoscaling_policy.scale_down	resource
aws_autoscaling_schedule.off	resource
aws_autoscaling_schedule.on	resource
aws_cloudwatch_metric_alarm.scale_down	resource
aws_iam_instance_profile.this	resource
aws_iam_policy.this	resource
aws_iam_role.this	resource
aws_iam_role_policy_attachment.this	resource
aws_iam_role_policy_attachment.user_defined_policies	resource
aws_instance.this	resource
aws_launch_template.this	resource
aws_security_group.this	resource
null_resource.validate_instance_profile	resource
aws_ami.ami	data source
aws_caller_identity.current	data source
aws_ec2_instance_type.this	data source
aws_ssm_parameter.this	data source

---

<!-- END_TF_DOCS -->