HashiQube with Traefik and Nomad/Vault Integration

About

This is a fork of the updated servian/hashiqube repo which uses the Vagrant Docker Provider, which enables users to run HashiQube on Mac M1 processors.

It includes the following modifications:

1. Update traefik.nomad to support gRPC endpoints on port 7233.
2. Configure Nomad to allow it to pull Docker images from private GitHub repos given a GitHub personal access token (PAT). (See item #2 in the Quickstart).
3. Configure Nomad/Vault integration so that you can use Nomad to pull secrets from Vault. See hashicorp/nomad.sh and hashicorp/vault.sh
4. Add an OpenTelemetry Collector job.
5. Add a sample 2048-game job.

!! WARNING !!

For backward compatibility, the version of HashiQube used in my HashiQube series of blog posts on Medium can be found on the main branch.

Pre-requisites

• Docker (version 20.10.17 at the time of this writing)
• Vagrant (version 2.3.1 at the time of this writing)
• A GitHub Personal Access Token (PAT)

Quickstart

To get started:

1. Clone the repo

   git clone git@github.com:avillela/hashiqube.git
   git checkout m1_support

2. Configure the Docker plugin

   Note: If you wish to skip this configuration, simply comment out lines 46–52 and lines 74–88 in nomad.sh.

   The nomad.sh file has some additional configuration which enables you to pull Docker images from a private GitHub repo. This is enabled in lines 46–52 in the docker stanza, telling it to pull your Docker repo secrets from /etc/docker/docker.cfg, which is configured in lines 74–88.

   Line 79 expects a GitHub auth token, which is made up of your GitHub username and GitHub PAT. It pulls that information from a file called secret.sh, located at vagrant/hashicorp/nomad on the guest machine (mapped to hashiqube/hashicorp/nomad on the host machine).

   For your convenience, you can create secret.sh on your host machine like this (assuming you're starting from the hashiqube repo root directory):

   cat hashicorp/nomad/secret.sh
   echo "export GH_USER=<your_gh_username>" > hashicorp/nomad/secret.sh
   echo "export GH_TOKEN=<your_gh_pat>" >> hashicorp/nomad/secret.sh

   Be sure to replace <your_gh_username> with your own GitHub username and <your_gh_pat> with your own GitHub PAT.

3. Start Vagrant

   NOTE: If you use the DOCKER_DEFAULT_PLATFORM flag, make sure that you unset it first, otherwise you will run into problems.

   cd hashiqube # if you aren't already there
   vagrant up --provision-with basetools,docker,vault,consul,nomad --provider docker

   Now wait patiently for Vagrant to provision and configure your Docker image.

   Once everything is up and running (this will take several minutes, by the way), you'll see this in the tail-end of the startup sequence, to indicate that you are good to go:

   

4. Access the Hashi tools

   The following tools are now accessible from your host machine

   • Vault: http://localhost:8200 (see Vault Setup for more on accessing Vault from the command-line)
   • Nomad: http://localhost:4646
   • Consul: http://localhost:8500
   • Traefik: http://traefik.localhost

   If you'd like to SSH into HashiQube, you can do so by running the following from a terminal window on your host machine.

   vagrant ssh

5. Install the Nomad and Vault CLIs on your host machine

   If you’re using a Mac, you can install the Vault and Nomad CLIs via Homebrew like this:

   brew tap hashicorp/tap
   brew install hashicorp/tap/vault
   brew install hashicorp/tap/nomad

   If you’re not using a Mac, you can find your OS-specific instructions for Vault here and for Nomad here. Note that these are binary installs, and they also contain the CLIs.

Vault Setup

To access Vault from the command-line, first, set the VAULT_ADDR environment variable:

export VAULT_ADDR=http://localhost:8200

Next, set the VAULT_TOKEN environment variable. To get this token, you will need to log into the guest machine:

vagrant ssh

Once inside the guest machine:

cat /etc/vault/init.file | grep Root | rev | cut -d' ' -f1 | rev > /vagrant/hashicorp/token.txt

This saves the token to a text file that is also available on your host machine at ./hashicorp/token.txt. (And yes, this file is .gitignored. 😉)

Now open a terminal on your host machine, and run the following:

export VAULT_TOKEN=$(cat hashicorp/token.txt) && \
rm hashicorp/token.txt

Notice how we deleted hashicorp/token.txt...just to be safe. 😉

NOTE: In real life, you would never use the root token to set VAULT_TOKEN. But we’re on our own dev environment, so it’s not the end of the world.

Now you're ready to access Vault from the command line!

Gotchas

Exposing gRPC port 7233

I exposed gRPC port 7233 in Trafik. In order for programs on the host machine to access services on port 7233 on Vagrant, we need to expose it in the Vagrantfile like this:

config.vm.network "forwarded_port", guest: 7233, host: 7233

DNS Resolution with *.localhost

If you're using a Mac and are running into issues getting your machine to resolve *.localhost, you need to manually add the following entries to /etc/hosts:

# For HashiQube
127.0.0.1   traefik.localhost
127.0.0.1   2048-game.localhost
127.0.0.1   otel-collector-http.localhost
127.0.0.1   otel-collector-grpc.localhost

Why do we use 127.0.0.1? Because the Docker image is available to the host machine via localhost. Which means that to use localhost subdomains (e.g. traefik.localhost), you need to associate the localhost address (i.e. 127.0.0.1) with them.

NOTE: You'll have to keep manually adding entries to /etc/hosts each time you need a specific *.localhost entry. For example, if I need foo.localhost to resolve, I would add this line to the end of /etc/hosts: 127.0.0.1 foo.localhost