PCF Bootstrap Environment

Bootstrap an environment with BOSH, secrets management (Hashicorp Vault), Concourse, and PCF. PCF is installed with the PCF Pipelines. Uses BOSH Boot Loader plan patches to simplify setup. Currently working with AWS and GCP.

Steps

1. Assure dependencies are met (see below)
2. If using GCP, enable the appropriate APIs.
3. Edit lib/env.sh to match your needs.
4. Run prepare to prepare infrastructure and the bootstrap BOSH environment
5. Make sure DNS for your bootstrap subdomain is delegated from your primary zone (if needed).
6. Run secrets to make Vault available for secrets.
7. Configure the secrets for the bootstrap environment by running configure.
8. Add concourse to the environment with concourse.
9. (optional) Add LDAP to the environment with ldap.
10. Set the environment variable PIVNET_TOKEN to your Pivotal Network API token.
11. An SSH key in your .ssh directory named concourse_github with the public key registered with your Github account.
12. Add secrets under concourse/pcf/deploy-pcf path for your Google Cloud Storage S3-compatible access key id (gcp_storage_access_key) and secret access key (gcp_storage_secret_key) in vault.
13. Running pcf now will load PCF Platform Automation with Concourse, nee PCF Pipelines, into the Concourse you just installed and trigger the right jobs to install PCF.

Using the environment

All your connections to BOSH will be through SSH tunnels to the Jumpbox that the BOSH Bootloader creates. To use the bosh CLI, make soure you source the file work/bbl-env.sh into your shell with . work/bbl-env.sh, which will set up the proxy that BOSH uses. If the tunnels times out, you can recreate it with prepare client login.

LDAP, Vault, Concourse, and PCF have load balancers. You can access them at the expected URIs based on your configuration. The PCF Pipelines are available in the concourse team pcf, with username pivotal. To get the password run pcf secret concourse

Getting rid of the environment

Each of the scripts has a teardown command-line argument (except prepare). Run those, then run teardown.

1. Teardown PCF (pcf teardown).
2. If you added LDAP, remove it from the environment with ldap teardown.
3. Take down concourse with concourse teardown.
4. Get rid of Vault with secrets teardown.
5. Lastly, take down the infrastructure with teardown.

Ergonomics

Each command has some subcommands for running a piece of what it does. More to come on that later.

Dependencies

1. BOSH Boot Loader 6.x or later. It's in the Cloud Foundry tap on Homebrew, so Mac users can run brew install cloudfoundry/tap/bbl.
2. Safe. On a Mac you can run brew install starkandwayne/cf/safe.
3. Hashicorp Vault CLI. If you're on a Mac run brew install vault.
4. The Pivotal Network CLI, pivnet. Again, with Homebrew brew install pivotal/tap/pivotal-cli.
5. Certbot to get certificates from Let's Encrypt. Install with brew install certbot.

Coming soon

1. Deploy concourse with standard manifest plus ops files from concourse-deployment 1. Make cloud config changes idempotent.
2. Use Credhub (created by bbl or standalone) instead of deploying/managing vault.
3. Windows in PCF and concourse
4. PCF tile support
5. Split working directory from script directory to simplify having local changes
6. Simple script(s) to do the manual stuff more easily.
7. Forcing and/or testing without SSH multiplexing. I use it all the time, colleagues who don't are seeing some weirdness.
8. Making this document more readable and useful.
9. Making secrets highly available.
10. Making LDAP highly available. 1. Other IaaSes.
11. Rewrite in a programming language for better modularity and invocation across modules