Genesis - A BOSH Deployment Paradigm
Genesis v2.0 is the first version of Genesis to fully support BOSH v2. It is primarily geared to deployments that make use of Cloud Config, and Runtime Config. The BOSH v2 CLI is also a requirement of Genesis v2.
Genesis v2.0 builds upon the previous generation of Genesis, eliminating the vast majority of YAML files all over the place, leading to confusion and questions like "Where do I put property X - properties.yml, networking.yml, or credentials.yml?"
It also supports the next generation of Genesis deployment templates - Kits. In the old genesis, deployment templates were pulled in once, forked from their upstream, and likely never reconciled. With kits, you can keep upgrading the kit, pulling in newer versions of your deployment to make life much easier down the road.
Credential Rotation Built-in
Genesis v2 makes use of Vault as a back-end for storing credentials, most easily deployed
- managed via the Vault kit, and
the safe CLI. Each kit will auto-generate credentials
where appropriate, ensuring that each environment has unique and secure credentials. Secrets
can be manually or automatically rotated at the drop of hat with the
genesis secretscommand. Kits will define certain credentials as fixed, indicating that they should not be rotated under normal circumstances, as that would have ill effects on the deployment (the CF db encryption key for example).
The New Tiered Architecture
In Genesis v2, the data that was previously stored in
site by and large go away.
global is provided via Kits. Most of
site is now moved into Cloud Config. As a result,
we can stick most of the customization into a single file per environment, and the directory
structure has been flattened. To share information between environments, and reduce config repitition,
you can create files based on shared prefixes of the environment. For example,
us-west prefixes, and could share configuration in files named as such).
Here's what the new layout looks like:
. ├── ci.yml ├── us-boshlite-alpha.yml ├── us-east-dev.yml ├── us-east-prod.yml ├── us.yml # shared with us-*.yml ├── LICENSE └── README.md
When using a kit with Genesis v2, Genesis will automatically download the latest (or
specified) version of a kit, when you initialize your deployment repo. Any new
versions of the kit can be retrieved with
genesis download. Each deployment environment
must have at some point in its merge-path, a
kit section, indicating what kit + version
should be deployed. It will look something like this:
--- kit: name: jumpbox version: 2.0
To get the full benefit of CI, operators should place this in a file that is shared across all environments, so that upgrades can be vetted in the deployment pipeline in non-production environments, before going to production.
Flexible Deployment Pipelines
The pipeline strategy of Genesis v2.0 is much more flexible than the previous approach. Operators are able to define what environments should trigger, which should not, as well as which environments are gateways to deploying in later environments. Stemcell management is built-in, as are locking mechanisms to ensure that your BOSH isn't upgraded while it's in the middle of deploying something.
For a full run-down on Genesis v2.0 + deployment pipelines, see our pipeline documentation
On Ubuntu/Debian you can install
genesis and all its dependencies:
wget -q -O - https://raw.githubusercontent.com/starkandwayne/homebrew-cf/master/public.key | apt-key add - echo "deb http://apt.starkandwayne.com stable main" | tee /etc/apt/sources.list.d/starkandwayne.list apt-get update apt-get install genesis -y
On OS X/Mac:
brew tap starkandwayne/cf brew tap cloudfoundry/tap brew install genesis spruce safe bosh-cli vault git
genesis requires Perl. But Perl is everywhere.
You will need to set up Git:
git config --global user.name "Your Name" git config --global user.email email@example.com
Here are a few thoughts on the Genesis CLI.
Global options should be recognized for all commands:
--debug- Enable debugging mode, wherein Genesis prints messages to standard error, detailing what tasks it is doing, what commands it is running, return values, etc.
--trace- Enable debugging mode in all called utilities, like
bosh, where available.
-C PATH- Perform all operations from
PATHas the current working directory.
--yes- Answer "yes" to all questions, automatically, on behalf of the user. This means, for example, running
bosh -n, instead of just
Initialize a Genesis repo:
genesis init --kit shield genesis init --kit shield/1.2.3
Create a new deployment named us-west-1-sandbox:
genesis new us-west-1-sandbox
Generate a manifest for an environment:
# Using the live the cloud-config from the BOSH director genesis manifest my-sandbox # or, using the local file cached-cloud-config.yml: genesis manifest -c cached-cloud-config.yml
Deploy a deployment, manually:
genesis deploy us-west-1-sandbox
Rotate credentials for a deployment:
genesis secrets us-west-1-sandbox
Summarize the current state of deployments and what kits they are using:
Download version 1.3.4 of the SHIELD kit:
genesis download shield 1.3.4
Deploy the Genesis CI/CD pipeline configuration to Concourse:
Describe a given pipeline, in words:
Draw a pretty picture of a pipeline, using
dot, suitable for
embedding in documentation and putting on your blog:
genesis graph pipelines/aws
Embed the calling Genesis script in the Genesis repo:
Transitioning to Genesis v2
Due to the sweeping changes involved in Genesis v2, it is recommended to start
with a fresh deployment repo, and migrate existing deployments slowly but surely
over to Genesis v2 using the appropriate kits, or if necessary, a
dev kit (see
Authoring Kits for more info). The
genesis command will
auto-detect if you are running inside a Genesis v1 repo, and switch to a
Design Notes + Genesis Developer Resources
Genesis v2.0 design documentation that was previously found here has moved.