Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Properly document how Buildkite + Bazel was setup #4

Open
thundergolfer opened this issue May 19, 2019 · 0 comments
Open

Properly document how Buildkite + Bazel was setup #4

thundergolfer opened this issue May 19, 2019 · 0 comments

Comments

@thundergolfer
Copy link
Owner

Quick sketch of what I remember doing:

  • Create a new pipeline with a single step that just runs bazel test //...
  • Go the Buildkite (BK) agents page and select the AWS Quickstart guide. Click the button that auto-creates a CloudFormation stack in your account.
  • After that CloudFormation (CF) stack has spun up, it's likely that you'll be able to see some agents appear in your agents list in the BK UI, but they won't be able to run the step yet.
  • Follow the instructions here on how to create an SSH keypair that BK and Github will share so that your private code can be pulled
  • Create a bootstrap script like the boostrap_agent.sh script found in this repo that is able to install Bazel and its requirements (e.g Java SDK) on the BK-default 'Amazon Linux2' AMIs.
  • Upload that script into a new S3 bucket next to the S3 buckets buildkite created (like the one for private SSH keys).
  • Go into IAM in the AWS console and copy the policy that grants the buildkite-agent role access to the SSH keys bucket and modify it so that it grants that role access to the bucket that contains your bootstrap script.
  • If this is working correctly, then when you re-run BK pipeline tasks you should see Bazel correctly starting.
  • Go into the CloudFormation stack and click the "Update" button to update some parameters:
  • Update MaxInstances to 1 (it's cheaper)
  • Update MinInstances to 0 (it's cheaper)
  • Set BuildkiteTerminateInstanceAfterJobDecreaseDesiredCapacity and
    BuildkiteTerminateInstanceAfterJobtotrue`. This will scale down the ASG instances when BK jobs are finished, which is cheaper
  • Reduce the EBS volume size to 100GB, which should be plenty when starting out
  • Up the instance from t2-nano to at least t2-small, because otherwise you'll get out of memory errors.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant