Scripts to automate a launch of a new Halyard VM and Spinnaker Cluster
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
resources
.gitignore
01_kube_up.sh
02_hal_up.sh
03_spin_up.sh
04_drone_up.sh
LICENSE
README.md

README.md

Slipway: Automated Halyard and Spinnaker provisioning

Slipway automates one use-case only: provisioning Spinnaker on a Kubernetes cluster, controlled by an Ubuntu host VM running Halyard on Google Cloud Platform based on Halyard on GKE Quickstart guide.

Slipway

Slipway also sets up Spinnaker to be publicly accessible with Oauth2 authentication and sets up some container registry repositories and a Slackbot. If this exact configuration is not what you want then the Spinnaker Setup docs might be a good starting point.

These scripts make a lot of decisions which might not fit your use-case. YMMV.

0. Preamble

Install the gcloud SDK then configure:

gcloud auth login
gcloud projects list
gcloud config set project $PROJECT_NAME

...where $PROJECT_NAME is the project you wish to provision infrastructure in.

1. Provision Kubernetes cluster

Get the Kubernetes provisioning script

curl -O  https://raw.githubusercontent.com/betandr/slipway/master/01_kube_up.sh

Run Kubernetes provisioning script

sh 01_kube_up.sh $INDEX $CLUSTER_NAME $IP_REGION $CLUSTER_ZONE

...where $INDEX is your index, such as 001, $CLUSTER_NAME is your cluster name such as foobarbaz, $IP_REGION is the region in which the external IPs for the Spinnaker external endpoints will be provisioned such as europe-west2, and $CLUSTER_ZONE is the zone you wish to provision in, such as europe-west2-c.

For these example values the following cluster will be provisioned:

  • foobarbaz-cluster-001: 6 Nodes of 2 vCPUs in Zone europe-west2-c

The script will also reserve two IP addresses spinnaker-$INDEX and spinnaker-api-$INDEX which you should note for the Run Spinnaker provisioning script step later.

2. Provision Halyard Host VM

Get the Halyard host provisioning script

curl -O  https://raw.githubusercontent.com/betandr/slipway/master/02_hal_up.sh

Run Halyard host provisioning script

sh 02_hal_up.sh $INDEX

This script will ask you if you want to connect to the provisioned instance but also creates a shell script which can be used to connect to that instance later on.

To connect, run:

sh connect-to-halyard-host-$INDEX.sh

...where $INDEX matches your instance.

IMPORTANT: ALL FURTHER SCRIPTS ARE RUN FROM THE HALYARD HOST VM!

3. Provision Kubernetes Spinnaker and production clusters (from Halyard host VM):

Set up gcloud on Halyard VM

gcloud auth login
gcloud projects list
gcloud config set project $PROJECT_NAME

...where $PROJECT_NAME is the project you wish to provision infrastructure in.

Set up a DNS record for Spinnaker UI and Spinnaker API

Follow the Public Spinnaker on GKE) instructions to reserve two IP addresses then create DNS records for the Spinnaker API and UI load-balancers. The DNS records become OVERRIDE_API_URL and OVERRIDE_UI_URL and the IPs are used to configure Kubernetes in the Run Spinnaker provisioning script step.

Get the Spinnaker provisioning script

curl -O  https://raw.githubusercontent.com/betandr/slipway/master/03_spin_up.sh

Set the environment variables:

Name Description Example
IP_REGION The region for the Spinnaker external IPs europe-west2
CLUSTER_ZONE Your cluster zone europe-west2-c
OAUTH2_DOMAIN Used to restrict Oauth2 logins to a domain example.com
OAUTH2_CLIENT_ID (Optional — if not used; no Oauth2) Oauth2 Client ID (see Public Spinnaker on GKE) 00000000000-a0a0a0a0a0a0a0a0a0a0a00a0a0.apps.googleusercontent.com
OAUTH2_CLIENT_SECRET Oauth2 Client Secret (see Public Spinnaker on GKE) A0A0A0A0A0A0A0A0A0A0-A0
OAUTH2_REDIRECT_URL Oauth2 redirect URL (see Public Spinnaker on GKE) http://spinnaker-api.example.com/login
BOTNAME (Optional — if not used; no Slack) The Slackbot name (set up in Slackbot). You will be prompted for the token. (see Slack: Bot Users) slackbot
OVERRIDE_API_URL Public Spinnaker API endpoint (see Public Spinnaker on GKE) http://spinnaker-api.example.com
OVERRIDE_UI_URL Public Spinnaker UI endpoint (see Public Spinnaker on GKE) http://spinnaker.example.com
SPIN_GATE_IP The spin-gate reserved IP from the Run Kubernetes provisioning script step 35.45.25.15
SPIN_DECK_IP The spin-deck reserved IP from the Run Kubernetes provisioning script step 35.45.25.15

One method to do this is to have a local secrets file, containing something like:

export INDEX=000
export CLUSTER_NAME=hello-world
export IP_REGION=europe-west2
export CLUSTER_ZONE=europe-west2-c
export OAUTH2_DOMAIN=example.com
...etc...

...then load these using:

source secrets

Add your repos:

To ensure Spinnaker is set up with all the repos it needs, you should create a file called repos.txt and inside have repositories such as:

project1/container-a
project2/container-b

Run Spinnaker provisioning script

sh 03_spin_up.sh $INDEX $CLUSTER_NAME

...where $INDEX and $CLUSTER_NAME match the values specified in Run Kubernetes provisioning script in the 1. Provision Kubernetes clusters section.

Enter reserved external addresses

To enable external access to Spinnaker the Kubernetes service config will be updated. You will be prompted for the spinnaker-$INDEX and the spinnaker-api-$INDEX addresses from the 1. Provision Kubernetes clusters step.

04. Drone.io

Run Drone provisioning script

Helm is used to install Drone and the script can also install cert-manager to manage certificates. To use TLS without cert-manager, you will need to set the TLS secret manually. The script will tell you if this needs to be done.

sh 04_drone_up.sh

Values in Use

Until we start using a better way of storing our current infrastructure state, here is a record of the current values used to provision Drone:

| Variable | Replacement Key in values.yaml | Current Value | Notes | |========================|==================================|========================|========================================================| | Project Name | X_PROJECT | datalabdrone | Prefix for k8s deployments, not the GCP project name | | Host Name | X_SERVER_HOST | drone.datalab.rocks | | | Drone admin username | X_DRONE_ADMIN | bbcdatalab | should match Github user name | | Github OAuth Client ID | X_DRONE_GITHUB_CLIENT | obtained from github | | | Github OAuth Secret | X_DRONE_GITHUB_SECRET | obtained from github | |

Decommissioning

This is not yet automated as it could potentially be dangerous, but things to clear up for the clusters and VM host are:

  • Compute Engine: Halyard Host VM: halyard-host-$INDEX + Persistent Disk
  • Kubernetes Engine: $CLUSTER_NAME-cluster-$INDEX
  • Service Account: halyard-service-account-$INDEX@$PROJECT.iam.gserviceaccount.com
  • Service Account: gcs-spin-service-account-$INDEX@$PROJECT.iam.gserviceaccount.com
  • IAM Permissions: gcs-spin-service-account-$INDEX@$PROJECT.iam.gserviceaccount.com
  • IAM Permissions: halyard-service-account-$INDEX@$PROJECT.iam.gserviceaccount.com
  • VPC Network: spinnaker-$INDEX reserved external IP
  • VPC Network: spinnaker-api-$INDEX reserved external IP
  • VPC Network: target pools, forwarding rules

TODO

  • Check quota before provisioning
  • Check permissions before provisioning
  • Get the reserved external IPs automatically via gcloud compute addresses list.