Skip to content
This repository has been archived by the owner on Aug 9, 2024. It is now read-only.

Commit

Permalink
Merge pull request #2 from amazeeio/read-only-replica
Browse files Browse the repository at this point in the history
Read replica support
  • Loading branch information
Schnitzel committed Oct 23, 2019
2 parents b1563ad + 24b04e4 commit 30fb037
Show file tree
Hide file tree
Showing 5 changed files with 278 additions and 29 deletions.
101 changes: 72 additions & 29 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
# MariaDB as a Service APB

This Ansible playbook bundle provisions users and databases on a existing MariaDB instance.
This Ansible Playbook Bundle (APB) provisions users and databases on a existing MariaDB instance.

## Installation

Set up the Ansible service broker to import APBs from the Docker Hub appuio repository:

```yaml
registry:
- name: amazeeiolagoon
Expand All @@ -13,52 +15,93 @@ registry:
white_list: [.*-apb$]
```
To provide the admin user credentials to connect to the MariaDB, two secrets with the name `dbaas-db-credentials-production` and `dbaas-db-credentials-development` needs to exist in the Ansible service broker namespace:
```yaml
apiVersion: v1
kind: Secret
metadata:
name: dbaas-db-credentials-production
type: Opaque
stringData:
mariadb_hostname: db.maria.com
mariadb_password: myPassword
mariadb_port: '3306'
mariadb_user: root
```
To provide the admin user credentials to connect to the MariaDB, a secret with the name `lagoon-dbaas-db-credentials` needs to exist in the Ansible service broker namespace:

```yaml
apiVersion: v1
kind: Secret
metadata:
name: dbaas-db-credentials-development
name: lagoon-dbaas-db-credentials
type: Opaque
stringData:
mariadb_hostname: db.maria.com
mariadb_password: myPassword
mariadb_port: '3306'
mariadb_user: root
production_mariadb_hostname: db.maria.com
production_mariadb_readreplica_hostname: db.readreplica.maria.com
production_mariadb_password: myPassword
production_mariadb_port: '3306'
production_mariadb_user: root
development_mariadb_hostname: db.maria.com
development_mariadb_readreplica_hostname: db.readreplica.maria.com
development_mariadb_password: myPassword
development_mariadb_port: '3306'
development_mariadb_user: root
```

If your environment has a read-replica mariadb endpoint, you can configure `*_mariadb_readreplica_hostname` with the read-replica hostname.
Otherwise, if there is no read-replica available, just populate it with the same value as `*_mariadb_hostname`.

The Ansible service broker needs to be configured to mount the secret in provisioner pods. Add the following section to the Ansible service broker configuration (ConfigMap):

```yaml
secrets:
- title: DBaaS database credentials
secret: dbaas-db-credentials-production
apb_name: lagoon-dbaas-mariadb-apb
- title: DBaaS database credentials
secret: dbaas-db-credentials-development
secret: lagoon-dbaas-db-credentials
apb_name: lagoon-dbaas-mariadb-apb
```

## Development environment
You can use [minishift](https://github.com/minishift/minishift) with the [Ansible Service Broker Addon](https://github.com/minishift/minishift-addons/tree/master/add-ons/ansible-service-broker) to run a local OpenShift installation with the Ansible service broker to test APBs:
```bash
MINISHIFT_ENABLE_EXPERIMENTAL=y minishift start --extra-clusterup-flags "--service-catalog" --openshift-version v3.9.0

minishift addons install <path_to_addon>
minishift addons apply ansible-service-broker
*NOTE: these scripts run `oc` commands, so don't run them while logged in to another cluster*

You can use [minishift](https://github.com/minishift/minishift) with the [Ansible Service Broker Addon](https://github.com/minishift/minishift-addons/tree/master/add-ons/ansible-service-broker) to run a local OpenShift installation with the Ansible service broker to test APBs.

The script `minishift-devel.sh` will set up a minishift development environment for you.
It requires these CLI tools to be installed:

* [oc](https://github.com/openshift/origin/releases)
* [apb](https://github.com/automationbroker/apb/releases)
* [helm](https://github.com/helm/helm/releases)

Also refer to the `apb` [developer documentation](https://github.com/automationbroker/apb/blob/master/docs/developers.md), and the other documents in that `/docs` directory.

### Tests

Basic integration tests can be run using `minishift-test.sh`, and assume an environment set up via the `minishift-devel.sh` script.
The tests require these CLI tools to be installed:

* [bats](https://github.com/bats-core/bats-core)
* [svcat](https://github.com/kubernetes-sigs/service-catalog/releases)

Example test output:

```
$ ./minishift-test.sh
✓ provision a service (development)
✓ bind the secret (development)
✓ check the contents of the secret (development)
✓ unbind the secret (development)
✓ deprovision the service (development)
✓ provision a service (production)
✓ bind the secret (production)
✓ check the contents of the secret (production)
✓ unbind the secret (production)
✓ deprovision the service (production)
10 tests, 0 failures
```

Install the [APB CLI](https://github.com/ansibleplaybookbundle/ansible-playbook-bundle/blob/master/docs/apb_cli.md#installing-the-apb-tool) on your machine. The easiest way is to run it in a Docker container via the provided [helper script](https://github.com/ansibleplaybookbundle/ansible-playbook-bundle/blob/master/scripts/apb-docker-run.sh).
### Local development workflow

```bash
# hack
...
# push
oc start-build -n openshift --follow --from-dir . dbaas-mariadb-apb
# test
svcat provision test-dbaas --class localregistry-dbaas-mariadb-apb --plan development --wait
svcat deprovision test-dbaas --class localregistry-dbaas-mariadb-apb --plan development --wait
```

## Release

An automatic Docker build is set up for this repository. If you change stuff in `apb.yml` don't forget to run `apb prepare` before committing.

1 change: 1 addition & 0 deletions defaults/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,4 @@ _apb_service_class_id: 0
_apb_service_instance_id: 0

app_name: "mariadb-{{ _apb_service_instance_id }}"
app_readreplica_name: "mariadb-readreplica-{{ _apb_service_instance_id }}"
102 changes: 102 additions & 0 deletions minishift-devel.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
#!/usr/bin/env bash
set -euo pipefail
set -x

if [[ -z "$KUBECONFIG" ]] || [[ $(realpath "$KUBECONFIG") = $(realpath "$HOME/.kube/config") ]]; then
echo -e '\nThis script runs oc commands, and you are using the global ~/.kube/config.' \
'If you are okay with this, hit enter to confirm.' \
'Or use an alternative $KUBECONFIG.\n'\
'e.g. export KUBECONFIG=$(mktemp --tmpdir kubeconfig.XXXXXXXX)\n'
read
fi

# start minishift with the appropriate components
minishift start --cpus=4 --memory=8GB
minishift openshift component add service-catalog
minishift openshift component add automation-service-broker

# give developer cluster-admin for apb builds/installs
oc login -u system:admin
oc adm policy add-cluster-role-to-user cluster-admin developer
oc login -u developer
# perform build
oc new-build -n openshift --binary=true --name dbaas-mariadb-apb
oc start-build -n openshift --follow --from-dir . dbaas-mariadb-apb

# wait for ASB to be deployed
while oc get pod -n openshift-automation-service-broker | grep deploy > /dev/null; do
sleep 5
done

# notify the broker to search for available APBs
# this executes asynchronously and may take a minute to finish
while ! apb --kubeconfig="$KUBECONFIG" broker bootstrap; do
sleep 5
done
# verify that the broker found the new APB
while ! apb --kubeconfig="$KUBECONFIG" broker catalog | grep dbaas-mariadb; do
sleep 5
done
# notify the service catalog web UI to update its catalog
apb --kubeconfig="$KUBECONFIG" catalog relist

# install the mariadb cluster
export TILLER_NAMESPACE=tiller
oc new-project $TILLER_NAMESPACE
export HELM_VERSION=v2.14.3 # get this from: helm -c --short
oc process -f https://github.com/openshift/origin/raw/master/examples/helm/tiller-template.yaml -p TILLER_NAMESPACE="${TILLER_NAMESPACE}" -p HELM_VERSION="${HELM_VERSION}" | oc create -f -
while ! oc get pod -n tiller | grep '1/1'; do
sleep 5
done
oc new-project mariadb-cluster
oc adm policy add-scc-to-user anyuid -z default
oc policy add-role-to-user cluster-admin "system:serviceaccount:${TILLER_NAMESPACE}:tiller"
helm install --name dbcluster stable/mariadb

# bind the secrets into deployment pods
updatedBrokerConfig=$(
# get the current configmap
oc -n openshift-automation-service-broker get cm broker-config -o json --export |
# pull out the broker-config field, which is raw YAML
jq -r '.data."broker-config"' |
# translate this YAML to JSON
ruby -ryaml -rjson -e 'puts YAML.load(ARGF).to_json' |
# append the secrets config to this JSON
jq '. += {secrets: [{title: "DBaaS database credentials", secret: "lagoon-dbaas-db-credentials", apb_name: "localregistry-dbaas-mariadb-apb"}]}' |
# convert back to YAML
ruby -ryaml -rjson -e 'puts JSON.load(ARGF).to_yaml' |
# escape the double quotes in preparation for insertion back into the configmap
sed 's/"/\\"/g'
)
# replace the existing configmap with the new one containing the secrets binding
oc -n openshift-automation-service-broker get cm broker-config -o json --export | jq -r ".data.\"broker-config\" = \"$updatedBrokerConfig\"" | oc -n openshift-automation-service-broker replace -f -
# rollout the service with the new configmap
oc -n openshift-automation-service-broker rollout latest dc/openshift-automation-service-broker
# wait on the rollout
oc -n openshift-automation-service-broker rollout status dc/openshift-automation-service-broker

mariadb_root_password=$(oc -n mariadb-cluster get secret dbcluster-mariadb -o json | jq -r '.data."mariadb-root-password"' | base64 -d)

# insert the required secrets into the right place
oc -n openshift-automation-service-broker apply -f - <<EOF
---
apiVersion: v1
kind: Secret
metadata:
name: lagoon-dbaas-db-credentials
type: Opaque
stringData:
production_mariadb_hostname: dbcluster-mariadb.mariadb-cluster.svc.cluster.local
production_mariadb_readreplica_hostname: dbcluster-mariadb.mariadb-cluster.svc.cluster.local
production_mariadb_password: $mariadb_root_password
production_mariadb_port: '3306'
production_mariadb_user: root
development_mariadb_hostname: dbcluster-mariadb.mariadb-cluster.svc.cluster.local
development_mariadb_readreplica_hostname: dbcluster-mariadb.mariadb-cluster.svc.cluster.local
development_mariadb_password: $mariadb_root_password
development_mariadb_port: '3306'
development_mariadb_user: root
EOF

# switch back to myproject
oc project myproject
93 changes: 93 additions & 0 deletions minishift-test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
#!/usr/bin/env bats

provision() {
run svcat provision test-dbaas \
--class localregistry-dbaas-mariadb-apb \
--plan $1 \
--wait
echo "$output"
[[ $status -eq 0 ]]
}

bind() {
run svcat bind test-dbaas \
--name test-dbaas-binding \
--secret-name test-dbaas-secret \
--wait
echo "$output"
[[ $status -eq 0 ]]
}

check_secret() {
run bash -c '
set -euo pipefail
data=$(oc get secret test-dbaas-secret -o json --export | jq -e ".data")
echo Secret data:
echo "$data"
echo "$data" | jq -e "select(
.DB_HOST? and
.DB_READREPLICA_HOST? and
.DB_NAME? and
.DB_PASSWORD? and
.DB_PORT? and
.DB_TYPE? and
.DB_USER?
)"
'
echo "$output"
[[ $status -eq 0 ]]
}

unbind() {
run svcat unbind test-dbaas \
--wait
echo "$output"
[[ $status -eq 0 ]]
}

deprovision() {
run svcat deprovision test-dbaas \
--wait
echo "$output"
[[ $status -eq 0 ]]
}

@test "provision a service (development)" {
provision development
}

@test "bind the secret (development)" {
bind
}

@test "check the contents of the secret (development)" {
check_secret
}

@test "unbind the secret (development)" {
unbind
}

@test "deprovision the service (development)" {
deprovision
}

@test "provision a service (production)" {
provision production
}

@test "bind the secret (production)" {
bind
}

@test "check the contents of the secret (production)" {
check_secret
}

@test "unbind the secret (production)" {
unbind
}

@test "deprovision the service (production)" {
deprovision
}
10 changes: 10 additions & 0 deletions tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
- name: read DB credentials
set_fact:
mariadb_hostname: "{{ lookup('file', secret ~ '_mariadb_hostname') }}"
mariadb_readreplica_hostname: "{{ lookup('file', secret ~ '_mariadb_readreplica_hostname') }}"
mariadb_port: "{{ lookup('file', secret ~ '_mariadb_port') }}"
mariadb_user: "{{ lookup('file', secret ~ '_mariadb_user') }}"
mariadb_password: "{{ lookup('file', secret ~ '_mariadb_password') }}"
Expand All @@ -30,6 +31,7 @@
DB_TYPE: "{{ 'mariadb' }}"
DB_NAME: "{{ create_db_name }}"
DB_HOST: "{{ app_name }}"
DB_READREPLICA_HOST: "{{ app_readreplica_name }}"
DB_PORT: "{{ mariadb_port }}"
DB_USER: "{{ create_db_user }}"
DB_PASSWORD: "{{ create_db_password }}"
Expand Down Expand Up @@ -80,6 +82,14 @@
type: ExternalName
external_name: "{{ mariadb_hostname }}"

- name: "{{ apb_action }} OpenShift readreplica service"
k8s_v1_service:
state: "{{ ensure_state }}"
name: "{{ app_readreplica_name }}"
namespace: "{{ namespace }}"
type: ExternalName
external_name: "{{ mariadb_readreplica_hostname }}"

- name: "{{ apb_action }} secret with DB credentials"
k8s:
state: "{{ ensure_state }}"
Expand Down

0 comments on commit 30fb037

Please sign in to comment.