Buildkite Elastic CI Stack for AWS
Buildkite is a platform for running fast, secure, and scalable continuous integration pipelines on your own infrastructure.
The Buildkite Elastic CI Stack gives you a private, autoscaling Buildkite Agent cluster. Use it to parallelize legacy tests across hundreds of nodes, run tests and deployments for all your Linux-based services and apps, or run AWS ops tasks.
Or jump straight in:
The current release is . See Releases for older releases.
Although the stack creates its own VPC by default, we highly recommend following best practice by setting up a separate development AWS account and using role switching and consolidated billing — see the Delegate Access Across AWS Accounts tutorial for more information.
aws cloudformation create-stack \ --output text \ --stack-name buildkite \ --template-url "https://s3.amazonaws.com/buildkite-aws-stack/latest/aws-stack.yml" \ --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM \ --parameters "$(cat config.json)"
This repository hasn't been reviewed by security researchers so exercise caution and careful thought with what credentials you make available to your builds.
Anyone with commit access to your codebase (including third-party pull-requests if you've enabled them in Buildkite) will have access to your secrets bucket files.
Also keep in mind the EC2 HTTP metadata server is available from within builds, which means builds act with the same IAM permissions as the instance.
To get started with customizing your own stack, or contributing fixes and features:
# Checkout all submodules git submodule update --init --recursive # Build all AMIs and render a cloud formation template - this requires AWS credentials (in the ENV) # to build an AMI with packer make build # To create a new stack on AWS using the local template make create-stack # You can use any of the AWS* environment variables that the aws-cli supports AWS_PROFILE="some-profile" make create-stack # You can also use aws-vault or similar aws-vault exec some-profile -- make create-stack
If you need to build your own AMI (because you've changed something in the
packer directory), run packer with AWS credentials in your shell environment:
This will boot and image three AWS EC2 instances in your account’s
- Linux amd64
- Linux arm64
- Windows amd64
We provide support for security and bug fixes on the current major release only.
If there are any changes in the master branch since the last tagged release, we aim to publish a new tagged release of this template at the end of each month.
We support all AWS Regions, except China and US GovCloud.
We aim to support new regions within one month of general availability.
We build and deploy the following AMIs to all our supported regions:
- Amazon Linux 2 (x86_64)
- Amazon Linux 2 (arm64)
- Windows Server 2019 (x86_64)
The Elastic CI Stack template published from the master branch tracks the latest Buildkite Agent release.
You may wish to preview any updates to your stack from this template using a CloudFormation Stack Change Set to decide whether to apply it.
Questions and support
Feel free to drop an email to firstname.lastname@example.org with questions. It helps us if you can provide the following details:
# List your stack parameters aws cloudformation describe-stacks --stack-name MY_STACK_NAME \ --query 'Stacks.Parameters.[ParameterKey,ParameterValue]' --output table
Provide us with logs from CloudWatch Logs:
You can also drop by
#aws channels in Buildkite Community Slack and ask your question!
See Licence.md (MIT)