Buildkite Agent Swarm

BEES for short: BuildkitE agEnt Swarm.

An unofficial and opinionated Buildkite Helm chart. Use this at your own risk etc. Support is best-effort.

Goals

• Turn-key deployment for docker-compose pipelines
• Fast and efficient builds
  • Minimise unnecessary overhead from duplicate work (builds, fetching, booting dependencies)
  • Maximising use of compute
    • Spread work across nodes

See also

• Buildkite's official agent-stack-k8s
  • Current implementation does not appear to work with docker-compose-buildkite-plugin
• Buildkite's official but deprecated Helm charts, which this is forked from.

Features

• Deploy multiple sets of agents with a single chart, with per-set:
  • queues
  • nodeSelectors
  • resources
  • volumeMounts
  • tags
• Ability to deploy Docker in Docker as a DaemonSet to minimise overhead on building and moving images between agents
• Spread agents evenly across nodes using pod anti-affinity
• Spread jobs evenly across nodes by using agent priority by inverting pod indexes
• Usability improvements:
  • Agents have node=<node> tags
  • Agent default name format: [prefix]<node>-<queue>-<pod suffix>

Roadmap / Ideas

• Built-in registry pull-through cache
• Docker image cleanup

Getting Started

More detailed instructions coming later.

1. Use Helm to deploy to your cluster. See values.yaml and tweak accordingly

Thanks ❤️

• A massive thanks to Rimas Mocevicius for authoring Buildkite Agent chart.

Copyright

Copyright (c) 2020-2023 Buildkite Pty Ltd, chendo. See LICENSE for details.