Permalink
Browse files

Update Docker and Kubernetes scripts (#861)

* wip: update deploy scripts

* remove old mysql dockerfile

* wip: waitforit initialize default domain

* Wait-for dockerfile

* Updated compose file

* Update readme

* remove old scripts

* cleanup

* Update docker-compose

This updates docker-compose so that it supports a single-line cluster
bring up.
- docker-compose up -d
- Auto initializes with an init container
- Changed service names to have `-` rather than `_` to be kubernetes
compatable.
- docker-compose is now fully kompose compatible.

* travis nits

* Add :latest tag to images

* nits

* Add go:generate for kubernetes scripts

* Better README info
  • Loading branch information...
gdbelvin committed Nov 24, 2017
1 parent c4b78d6 commit 808d4b9d70231ecdc49a0da4a01908d3817ecc7e
Showing with 697 additions and 648 deletions.
  1. +1 −2 .travis.yml
  2. +16 −15 README.md
  3. +1 −1 cmd/keytransparency-monitor/Dockerfile
  4. +1 −1 cmd/keytransparency-sequencer/Dockerfile
  5. +2 −2 cmd/keytransparency-server/Dockerfile
  6. +9 −0 deploy/docker/init/Dockerfile
  7. +2 −2 deploy/kubernetes/README.md
  8. +36 −0 deploy/kubernetes/db-deployment.yaml
  9. +0 −44 deploy/kubernetes/db-deployment.yml
  10. +19 −0 deploy/kubernetes/db-service.yaml
  11. +17 −0 deploy/kubernetes/gen.go
  12. +21 −0 deploy/kubernetes/init-pod.yaml
  13. +0 −158 deploy/kubernetes/keytransparency-deployment.yml.tmpl
  14. +43 −0 deploy/kubernetes/log-server-deployment.yaml
  15. +22 −0 deploy/kubernetes/log-server-service.yaml
  16. +46 −0 deploy/kubernetes/log-signer-deployment.yaml
  17. +19 −0 deploy/kubernetes/log-signer-service.yaml
  18. +43 −0 deploy/kubernetes/map-server-deployment.yaml
  19. +22 −0 deploy/kubernetes/map-server-service.yaml
  20. +40 −0 deploy/kubernetes/monitor-deployment.yaml
  21. +19 −0 deploy/kubernetes/monitor-service.yaml
  22. +27 −0 deploy/kubernetes/prometheus-deployment.yaml
  23. +19 −0 deploy/kubernetes/prometheus-service.yaml
  24. +35 −0 deploy/kubernetes/sequencer-deployment.yaml
  25. +19 −0 deploy/kubernetes/sequencer-service.yaml
  26. +48 −0 deploy/kubernetes/server-deployment.yaml
  27. +24 −0 deploy/kubernetes/server-service.yaml
  28. +0 −130 deploy/kubernetes/trillian-deployment.yml
  29. +0 −9 deploy/mysql-trillian/Dockerfile
  30. +61 −46 docker-compose.yml
  31. +0 −59 scripts/configure_trillian.sh
  32. +6 −118 scripts/deploy.sh
  33. +0 −6 scripts/gen_server_keys.sh
  34. +0 −15 scripts/log_payload.json
  35. +0 −15 scripts/map_payload.json
  36. +0 −25 scripts/prepare_server.sh
  37. +79 −0 scripts/wait-for.sh
View
@@ -49,15 +49,14 @@ before_deploy:
- tar -xzf travis_secrets.tar.gz
- gcloud auth activate-service-account --key-file service_key.json
- rm -f service_key.json
- gcloud --quiet version
- gcloud --quiet components update kubectl
- gcloud --quiet version
- gcloud config set project key-transparency
- gcloud config set compute/zone us-central1-a
- gcloud container clusters get-credentials ci-cluster
- go get github.com/google/trillian/server/trillian_log_server
- go get github.com/google/trillian/server/trillian_log_signer
- go get github.com/google/trillian/server/trillian_map_server
- go get github.com/google/trillian/cmd/createtree
deploy:
skip_cleanup: true
View
@@ -31,7 +31,7 @@ development.
## Key Transparency Client
### Setup
1. Install [Go 1.7](https://golang.org/doc/install).
1. Install [Go 1.9](https://golang.org/doc/install).
2. `go get -u github.com/google/keytransparency/cmd/keytransparency-client `
3. Get an [OAuth client ID](https://console.developers.google.com/apis/credentials) and download the generated JSON file to `client_secret.json`.
@@ -81,25 +81,26 @@ development.
1. `./scripts/prepare_server.sh -f`
### Run
1. Start Trillian
1. Run Key Transparency
```sh
$ docker-compose up -d trillian-map trillian-log
Creating keytransparency_db_1
Creating keytransparency_trillian-map_1
Creating keytransparency_trillian-log_1
$ docker-compose up -d
Creating keytransparency_db_1 ... done
Creating keytransparency_map_server_1 ... done
Creating keytransparency_log_server_1 ... done
Creating keytransparency_log_server_1 ... done
Creating keytransparency_server_1 ... done
Creating keytransparency_sequencer_1 ... done
Creating keytransparency_monitor_1 ... done
Creating keytransparency_init_1 ... done
Creating keytransparency_prometheus_1 ... done
Creating keytransparency_monitor_1 ... done
```
2. Provision a log and a map
```sh
source scripts/configure_trillian.sh && createLog && createMap
```
3. Run Key Transparency
- `docker-compose up -d`
2. Watch it Run
- `docker-compose logs --tail=0 --follow`
- [https://localhost:8080/v1/users/foo@bar.com?app_id=app1](https://localhost:8080/v1/users/foo@bar.com?app_id=app1)
- [https://localhost:8080/v1/domain/info](https://localhost:8080/v1/domain/info)
- [Proof for app1/foo@bar.com](https://localhost:8080/v1/domains/default/users/foo@bar.com?app_id=app1)
- [Server configuration info](https://localhost:8080/v1/domains/default/info)
- [Prometheus graphs](http://localhost:9090/graph)
## Development and Testing
@@ -1,4 +1,4 @@
FROM golang
FROM golang:1.9
ADD keytransparency/genfiles/* /kt/
ADD ./keytransparency /go/src/github.com/google/keytransparency
@@ -1,4 +1,4 @@
FROM golang
FROM golang:1.9
ADD ./keytransparency /go/src/github.com/google/keytransparency
ADD ./trillian /go/src/github.com/google/trillian
@@ -1,4 +1,4 @@
FROM golang
FROM golang:1.9
ADD keytransparency/genfiles/* /kt/
ADD ./keytransparency /go/src/github.com/google/keytransparency
@@ -11,4 +11,4 @@ RUN go get -tags="mysql" ./cmd/keytransparency-server
# See the README.md file on how to use docker-compose.
ENTRYPOINT ["/go/bin/keytransparency-server"]
EXPOSE 8080
EXPOSE 8080
@@ -0,0 +1,9 @@
FROM alpine:3.4
RUN apk add --update curl && \
rm -rf /var/cache/apk/*
ADD ./scripts/ /scripts
ENTRYPOINT ["/scripts/wait-for.sh"]
CMD ["--help"]
@@ -15,10 +15,10 @@ After that you should be able to visit http://localhost:8080/ui .
## Delete mysql DB
To delete all data written to the DB you can run:
```
kubectl delete -f deploy/kubernetes/db-deployment.yml
kubectl delete -f deploy/kubernetes/db-deployment.yaml
```
Alternatively, you use the kubernetes HTTP user interface:
1) Start a proxy to the kubernetes api by running `kubectl proxy --port=8080`.
2) Find and delete the mysql pod/deployment on the
[user interface](http://localhost:8080/ui).
@@ -0,0 +1,36 @@
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
annotations:
kompose.cmd: kompose convert --file ../../docker-compose.yml
kompose.version: 1.4.0 (HEAD)
creationTimestamp: null
labels:
io.kompose.service: db
name: db
spec:
replicas: 1
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
io.kompose.service: db
spec:
containers:
- env:
- name: MYSQL_DATABASE
value: test
- name: MYSQL_PASSWORD
value: zaphod
- name: MYSQL_RANDOM_ROOT_PASSWORD
value: "yes"
- name: MYSQL_USER
value: test
image: us.gcr.io/key-transparency/db
name: db
ports:
- containerPort: 3306
resources: {}
restartPolicy: Always
status: {}

This file was deleted.

Oops, something went wrong.
@@ -0,0 +1,19 @@
apiVersion: v1
kind: Service
metadata:
annotations:
kompose.cmd: kompose convert --file ../../docker-compose.yml
kompose.version: 1.4.0 (HEAD)
creationTimestamp: null
labels:
io.kompose.service: db
name: db
spec:
ports:
- name: "3306"
port: 3306
targetPort: 3306
selector:
io.kompose.service: db
status:
loadBalancer: {}
View
@@ -0,0 +1,17 @@
// Copyright 2017 Google Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package kubernetes
//go:generate kompose convert --file ../../docker-compose.yml
@@ -0,0 +1,21 @@
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
io.kompose.service: init
name: init
spec:
containers:
- args:
- sequencer:8080
- --
- curl
- -k
- https://sequencer:8080/v1/domains
- -d{"domain_id":"default","min_interval":"1s","max_interval":"60s"}
image: us.gcr.io/key-transparency/init:latest
name: init
resources: {}
restartPolicy: OnFailure
status: {}
Oops, something went wrong.

0 comments on commit 808d4b9

Please sign in to comment.