Table of Contents
- About the repo
- Quick start
- Repository structure
- CI/CD example with Gitlab CI and Helm
About the repo
Terraform is used to describe the desired state of the infrastructure, thus implementing Infrastructure as Code (IaC) approach.
Google Kubernetes Engine (GKE) service is used for cluster deployment. Since Google announced that they had eliminated the cluster management fees for GKE, it became the safest and cheapest way to run a Kubernetes cluster on GCP, because you only pay for the nodes (compute instances) running in your cluster and Google abstracts away and takes care of the master control plane.
Prerequisite: make sure you're authenticated to GCP via gcloud command line tool using either default application credentials or service account with proper access.
Check terraform.tfvars.example file inside
my-cluster folder to see what variables you need to define before you can use terraform to create a cluster.
You can run the following command in
my-cluster to make your variables definitions available to terraform:
$ mv terraform.tfvars.example terraform.tfvars # variables defined in terraform.tfvars will be automatically picked up by terraform during the run
Once the required variables are defined, use the commands below to create a Kubernetes cluster:
$ terraform init $ terraform apply
After the cluster is created, run a command from terraform output to configure access to the cluster via
kubectl command line tool. The command from terraform output will be in the form of:
$ gcloud container clusters get-credentials my-cluster --zone europe-west1-b --project example-123456
├── accounts │ └── service-accounts ├── my-cluster │ ├── deploy-app-example │ └── k8s-config │ ├── charts │ │ └── gitlab-omnibus │ │ ├── charts │ │ │ └── gitlab-runner │ │ │ └── templates │ │ └── templates │ │ ├── fast-storage │ │ ├── gitlab │ │ ├── ingress │ │ └── load-balancer │ │ └── nginx │ ├── env-namespaces │ ├── kube-lego │ └── storage-classes └── terraform-modules ├── cluster ├── firewall │ └── ingress-allow ├── node-pool └── vpc
The folder contains reusable pieces of terraform code which help us manage our configuration more efficiently by avoiding code repetition and reducing the volume of configuration.
The folder contains 4 modules at the moment of writing:
clustermodule allows to create new Kubernetes clusters.
firewall/ingress-allowmodule allows to create firewall rules to filter incoming traffic.
node-poolmodule is used to create Node Pools which is mechanism to add extra nodes of required configuration to a running Kubernetes cluster. Note that nodes which configuration is specified in the
clustermodule become the default node pool.
vpcmodule is used to create new Virtual Private Cloud (VPC) networks.
Inside the my-cluster folder, I put terraform configuration for the creation and management of an example of Kubernetes cluster. Important files here:
main.tfis the place where we define main configuration such as creation of a network for our cluster, creation of the cluster itself and node pools.
firewall.tfis used to describe the firewall rules regarding our cluster.
dns.tfis used to manage Google DNS service resources (again with regards to the services and applications which we will run in our cluster).
static-ips.tfis used to manage static IP addresses for services and applications which will be running in the cluster.
terraform.tfvars.examplecontains example terraform input variables which you need to define before you can start creating a cluster.
outputs.tfcontains output variables
variables.tfcontains input variables
k8-confingfolder contains Kubernetes configuration files (manifests) which are used to define configuration of the running Kubernetes cluster. It has 4 subdirectories inside:
env-namespacescontains manifests for creating namespaces, or virtual environments within the cluster, for running our services. In this example,
raddit-namespaces.ymlfile is used to describe 3 namespaces:
raddit-prodfor running example application (which is called raddit in this case) in different virtual environments, and
infranamespace for running services vital to our infrastructure like CI/CD, monitoring, or logging software.
storage-classesfolder is used to create storage classes that could be then used in dynamic volume provisioning for our applications.
kube-legofolder has the configuration required to run kube-lego service which is used for automatic SSL certificates requests for our services running inside the cluster.
chartscontains Helm charts for deploying infra services. In this case it only has a chart for deploying Gitlab CI along with a Runner.
deploy-app-examplehas an bunch of Kubernetes objects definitions which are used to deploy nginx to a Kubernetes cluster. You can use the command below to deploy nginx to the cluster once it is created:
$ kubectl apply -f ./deploy-app-example/nginx-example.yml
This is another top level folder in this project. It has a separate set of terraform files which are used to manage access accounts to our clusters. For example, you may want to create a service account for your CI tool to allow it to deploy applications to the cluster.
CI/CD example with Gitlab CI and Helm
For an example of building a CI/CD pipeline with Kubernetes, Gitlab CI, and Helm see this blog post.