Create your own local environment for trying out auto-generated TLS certificates. Test environment is run on a Debian Stretch machine on top of VirtualBox. This environment suitable for testing out Nomad and Vault PKI, but also HTTP/2 and service orchestration in general.
This environment has the following services built in:
- Consul server for storage backend and service discovery
- Vault server for PKI
- Nomad client & server
- Docker for running Nomad jobs
- Fabio running in a container as a load balancer
- Sample Go application running in a container. Available on Dockerhub.
Provisioning is handled by Ansible. Ansible calls Terraform to set up Vault and Consul configuration, which cannot be configured through config files.
provisioning/: Ansible configuration
group_vars: Global variables for selecting Vault/Nomad/Terraform/Consul versions
roles: Ansible roles for each component
jobs: Nomad jobs for running load balancer and app
tf: Terraform configuration
PKI is initialized as follows:
- Root CA is issued by Terraform.
- Vault PKI is initialized by creating a CSR for a sub-CA.
- Terraform signs the certificate request for sub-CA.
- Vault installs sub-CA certificate to its PKI.
Sub-CA private key is never exported and actually cannot even be exported from Vault.
How do I use this?
Ensure you have VirtualBox and Vagrant installed. Then run
vagrant up. During boot up
Ansible will print out Vault unseal keys and root token. Store them somewhere as you
will need them. This is the only time the keys are printed out and after that your
keys are unrecoverable.
Log in to the instance by running
vagrant ssh. Then run
nomad status to see which
services are scheduled to the instance.
$ nomad status ID Type Priority Status Submit Date app service 50 running 03/24/18 15:07:36 GMT lb service 50 running 03/24/18 15:05:12 GMT
After Ansible has finished provisioning you then have the following two Nomad jobs running:
appis a Go sample application which prints "welcome" and gives HTTP 200 as a response. It registers to Consul by service name "testapp".
lb: load balancer which uses healthy Consul services as backend and adds them automatically to its configuration. It also auto-generates TLS certificates and also renews them.
Verify that the setup is working by running the following command:
$ curl -vk https://testapp-lb.query.consul ... * Server certificate: * subject: CN=testapp-lb.query.consul * start date: Mar 24 15:07:35 2018 GMT * expire date: Apr 3 15:08:05 2018 GMT * issuer: CN=Sub-CA for HashiStack PKI ... * Using HTTP2, server supports multi-use ... < HTTP/2 200 < content-type: text/plain; charset=utf-8 < date: Sat, 24 Mar 2018 15:27:54 GMT < content-length: 7 < * Connection #0 to host testapp-lb.query.consul left intact welcome
From this we can see that our load balancer supports HTTP/2 and is serving
content by using TLS certificate with a Common Name (CN) of
You can also run the command from the host machine by using correct SNI hostname mapping.
$ curl -vk --resolve testapp-lb.query.consul:443:192.168.33.102 https://testapp-lb.query.consul
Finally check the load balancer logs:
$ nomad status lb | grep ' run ' dc41d70f 38334ef9 fabio 0 run running 25m29s ago 25m26s ago $ nomad logs -stderr dc41d70f ... + route add testapp testapp-lb.query.consul/ https://192.168.33.102:30819 opts "tlsskipverify=true" ... 2018/03/24 15:08:05 [INFO] cert: vault: issued cert for testapp-lb.query.consul; serial = 59-47-2b-38-c6-c3-88-37-22-f5-06-61-fd-68-9e-f1-5b-94-26-ac
Done, you made it. Have a beer.