Shipyard

This is my knowledge repo for containerized, scalable, monitored, and generally production ready web services.

The main components:

• A JSON RESTful API server, in Go
• A rudimentary and insecure identity provider, in Go
• A simple (and poorly styled) ReactJS web dashboard frontend
• A docker-compose config to coordinate the full system, with the addition of a Postgres database, a Prometheus server, and a Grafana server
• A Kubernetes config to orchestrate the full system, with the addition of a Postgres database, a Prometheus server, and a Grafana server

As my understanding of these services grow and I learn new things, I plan to add back to this repo with better practices and more idiomatic methods so that it may be a point of reference for me, myself, and I.

Ship It

There are currently 3 ways to stand up this project:

1. Manually.
2. Using docker-compose to build and run each Dockerized container
3. Using K8s with Minikube to run everything on a single node.

1 - Serving Manually

cd go/src/shipyard && make build-no-dbx && cd ../../..
cd web/shipyard && make build && cd ../..

You'll then need to manage how you run and connect your shipyard API binary, the web build, the database, and any potential monitoring.

2 - Serving with docker-compose

cd dockercompose && make build up

The default configuration will serve:

• The web frontend at localhost:3000
• The backend API at localhost:8000
• The Grafana metrics dashboard at localhost:5000

3 - Serving with Kubernetes and Minikube

# 1. make sure minikube is running
# 2. make sure necessary hosts are added to /etc/hosts on host machine and the
#    host minikube node (check k8s/README.md for more details)
cd k8s && make applydev

The default configuration will serve:

• The web frontend at shipyard.tech
• The backend API at api.shipyard.tech
• The Grafana metrics dashboard at grafana.shipyard.tech

How to Use

• Go to the web dashboard
• "Sign Up" to create a new user
• "Make Dummy Address"
• "Make Dummy Item"
• See the item in the marketplace
• Add the item to your cart
• Add the item to your cart again
• Order the items in your cart
• Logout
• See the Dummy items in the marketplace and no cart

Known Weak Points (TODOs)

• No SSL support yet. TODO = setup LetsEncrypt bot for docker-compose and K8s.
• The backend API src uses a handful of custom built tools that are essentially boiler plate. I would like to be able to easily reuse and improve these boiler plate tools. They should be broken out into their own library.
• The database is included in the docker-compose and K8s configurations for simplicity. IRL, the DB would live outside the cluster.
• There is no support for file serving.
• The Go database layer is using DBX, which seems to be unmaintained and is currently failing to build. A better database ORM should be used instead.
• The idp Go package was build ad-hoc. It's expected that it can be replaced entirely with a real auth service, like Auth0, but this needs to be tested.
• The K8s recipes should make better use of labels and selectors to organize services and ease command line instructions. Use things like app, tier, env, release, etc.
• In the K8s cluster, he Prometheus and Grafana services could potentially leverage Helm charts. But I am unfamiliar with Helm, at the moment.
• In the K8s cluster, the service DNS could be handled better - instead of adding hostnames to /etc/hosts. Look into using the FQDNs.

Useful Links/Ideas/Notes:

• How to use local docker images with a minikube cluster https://medium.com/bb-tutorials-and-thoughts/how-to-use-own-local-doker-images-with-minikube-2c1ed0b0968
• Mapping to external services, like communicating with an external db https://cloud.google.com/blog/products/gcp/kubernetes-best-practices-mapping-external-services