ks8-2.md

Kubernetes series part 8-2

The objective of this tutorial is to prevent losing the data from our todo list application. As we saw in the ks8-1 episode, if you delete the database pod all data is lost. We'll fix this by using a database that lives externally to our cluster.

Even though the database is outside of the Kubernetes cluster, we'll still take advantage of Kubernetes services to allow our Python web application to connect to the external service.

Note that the code changes in this episode are based off of ks8-1.

Code changes

1. Remove the ks.database.deployment.yaml since we won't use a database in a cluster in this episode.

   ➜ pwd
       ~/dev/github/redgate/ks/ks8-2
   ➜ rm ./ks/templates/ks.database.deployment.yaml

2. Use Docker to start a PostgreSQL database container on your host machine

   We'll use Docker to avoid having to install the full PostgreSQL server on your host machine.

   In a separate terminal:

   # Make sure you're using the host docker machine for this container
   ➜ eval $(docker-machine env -u)
   ➜ docker pull postgres:9.5
   ➜ docker run -e POSTGRES_PASSWORD -e POSTGRES_USER=redgate -e POSTGRES_DB=ks -p 5432:5432 postgres:9.5

   NOTE: Make sure you use the same values from your Helm values.yaml file for the Postgres user, password and database in the above command.

   This will start a postgres docker container on your host machine, which you can connect to on localhost, port 5432. You can test this by trying to connect.

   We'll use pgcli as it has nice features like auto completion and syntax highlighting. (You can use the default PostgreSQL client psql if it's available on your machine - the syntax is the same)

   ➜ pgcli -h localhost -p 5432 -U redgate -w ks
   Version: 1.8.1
   
   redgate@localhost:ks> 

3. Create the necessary database table

   When you connect using pgcli, make sure you press F3 to enable multiline mode, otherwise the script below won't work

   ➜ pgcli -h localhost -p 5432 -U redgate -w ks
   
   redgate@localhost:ks> CREATE TABLE todo_list(
                         id SERIAL PRIMARY KEY,
                         task_id TEXT UNIQUE NOT NULL,
                         name TEXT NOT NULL,
                         done BOOLEAN NOT NULL);
   CREATE TABLE
   Time: 0.009s

4. Edit the ks.database.service.yaml to map to our local PostgreSQL installation

   We replace the contents of the file with the following:

   apiVersion: v1
   kind: Service
   metadata:
     creationTimestamp: null
     labels:
       run: {{ .Release.Name }}-{{ .Values.database.name }}-service
     name: {{ .Release.Name }}-{{ .Values.database.name }}-service
   spec:
     type: ExternalName
     externalName: {{ .Values.database.externalName }}

   Note that we still use the database service, to allow our Kubernetes pods to communicate to the external database without needing to know the details explicitly.

5. Add an 'externalName' entry to your Values.yaml file with the IP address used by minikube

   externalName: 192.168.64.1

   NOTE: The IP address used above is for the xhyve driver in Minikube. Depending on which driver you use, you will need to use a different IP address to connect out to the host machine.

   See here for more information

   Alternatively, if you are running this example in a cloud service, then you could point the external name to the hostname of a cloud-hosted database (e.g Amazon RDS, or equivalent).

Set up and start ks8-2

1. start minikube and build docker images

   ➜ cd ks8-2
   ➜ pwd
       ~/dev/github/redgate/ks/ks8-2
   ➜ minikube start
   ➜ eval $(minikube docker-env)
   
   # Ensure you've built the react app first
   ➜ cd app
   ➜ yarn
   ➜ yarn build
   ➜ cd ..
   
   ➜ docker build -f ./server/Dockerfile -t ks8webserverimage .
   ➜ docker build -f ./web/Dockerfile -t ks8webimage .

2. mount volume

   Run this command in a separate terminal

   ➜ cd ks8-2
   ➜ pwd
       ~/dev/github/redgate/ks/ks8-2
   ➜ minikube mount .:/mounted-ks8-src

3. install helm chart

   ➜ pwd
       ~/dev/github/redgate/ks/ks8-2
   ➜ helm install ./ks -n ks

4. check app is working properly

   ➜ kubectl get pods
   NAME                         READY     STATUS    RESTARTS   AGE
   ks-ks8web-5658c6fd94-vk5ms   2/2       Running   0          5m

5. check logs

   ➜ kubectl logs ks-ks8web-5658c6fd94-vk5ms ks8webfrontend
   ➜ kubectl logs ks-ks8web-5658c6fd94-vk5ms ks8webserver

6. test app in browser

   ➜ minikube service ks-ks8web-service

7. after inserting some data in the webserver, connect to the database and query the todo_list table

   ➜ pgcli -h localhost -p 5432 -U redgate -w ks
   
   redgate@localhost:ks> select * from todo_list;
   +------+---------------+--------+--------+
   | id   | task_id       | name   | done   |
   |------+---------------+--------+--------|
   | 1    | 1516804951842 | one    | True   |
   | 2    | 1516804953872 | four   | False  |
   | 3    | 1516804952568 | two    | True   |
   | 4    | 1516804953344 | three  | True   |
   +------+---------------+--------+--------+
   SELECT 4
   Time: 0.004s
   redgate@localhost:ks> 

   Even if we delete our Kubernetes application, our data will persist because it lives entirely separate to the cluster.

   Note that because we're using a docker container, we still have the problem of data persistence. If you stop your docker container for any reason, you will still lose the data. This can be overcome by mounting a volume from the host machine into your PostgreSQL container to store the data. That way, your data will persist outside of the container itself.

   This can be extended to a cloud scenario, as you could run a PostgreSQL Amazon RDS instance and connect your AWS Kubernetes cluster using the same external service configuration. This way, you get the benefits of RDS like high availability, automated backups and automatic minor version updates, among other things.

   However, one of the things you may have noticed is the need to manually run SQL Scripts to intialise the database. This obviously prone to error. We'll come back to managing database schema and migrations with Kubernetes in later episodes.