CRUD for Gitlab on Kubernetes ¹
This repo has Ansible roles and Terraform modules to set up a cluster running Gitlab CI and supporting services.
The setup and maintenance tasks are broken down into stages (cluster, server, service) and
verbs (create, ready, update, and eventually delete).
| Create | Ready | Update | Delete | |
|---|---|---|---|---|
| Cluster | ✅ | ⚠ | ✅ | ⚠ |
| Server | ✅ | ✅ | ✅ | ⚠ |
| Service | ✅ | 🚫 | ✅ | ⚠ |
WIP: this project works for me and I'd like to share, but may not work for you. Maybe not at all. Please open issues if you see anything, or just with questions or suggestions.
To get started, the docs directory has guides for:
This playbook sets up a kubernetes cluster in AWS, using kops, to run Gitlab and supporting tools.
Gitlab runners are set up with an autoscaling group of spot instances, which automatically shut down after 10 minutes idle and cost about 25% as much as on-demand instances.
Additional clusters can be included and the playbook is able to load context from kubeadm and kops.
Most of the tasks are system-independent and only require make and the tools used by the stage, but the
dependencies stage requires Ubuntu 16.04 or better. The latest Ubuntu LTS AMI is used for the kops nodes.
The following tools will be set up within the cluster:
- Gitlab (version control and CI)
- Gitlab runners (AWS spot instances and optionally dedicated servers)
- Cluster services
Using the AWS services:
- EC2 (on-demand, reserved, and/or spot instances)
- ElastiCache (redis)
- RDS (postgresql)
- Route53 (DNS)
With support from the SaaS tools:
- Foxpass (authentication/authorization)
- Grafana (graphing)
- Keybase (identity verification)
- Papertrail (logging)
- Sentry (error logging)
- UptimeRobot (availability monitoring)
make is used as a task runner, orchestrating the ansible playbook runs using tags. In short:
- you provide a config with DNS zone, VPC, and secrets
- a kubernetes cluster definition is rendered from the secrets into a terraform module.
- terraform runs to set up the cache, database, and kubernetes nodes.
- kubernetes service definitions are rendered from the secrets, config, and templates.
The following tools are used locally to set up the cluster:
Some of these should be installed before running, some can be installed by the Ansible playbook. Please see the setup guide § dependencies for details.
The version requirements are noted when a very recent version is needed, usually for a specific fix. Otherwise, most recent versions should work.
The resources created by this project will be in Amazon AWS.
You may provide your own kubernetes clusters (provisioned with kops or kubeadm) and allocate some services there, although the core services (Gitlab, DNS, and autoscaling) are only supported in AWS clusters.
TODO: provide a way to disable the kops cluster
This repo shows a way to automate build infrastructure and projects within it, using common open-source tools. This setup scales well from small, personal projects to company clusters.
¹: This project is not a replacement for reading the documentation. Kubernetes and Gitlab both offer excellent documentation, please read it.
Details of the services used here can be found at:
- Amazon EC2 Auto Scaling
- Amazon EC2 Spot Instances
- Gitlab CI Kubernetes Executor
- Gitlab CI YML
- kubernetes Concepts
- kubectl overview
- kops overview
This project is maintained by ssube.