example42 Puppet control-repo
A state of the art, feature rich, Puppet 4 control-repo. Use it to:
- Explore the layout of a modern, opinionated, general purpose, control-repo
- Create and use Puppet 4 optimised modules with updated design patterns
- Test your code in the Docker and Vagrant environments
- Fork and use as starting point for your Puppet infrastructure
- Manage the deployment workflow of your Puppet code
Download this repository:
git clone https://github.com/example42/control-repo cd control-repo
You need Puppet 4, possibly the latest release.
To install Puppet official repos and the latest Puppet agent run:
To setup the Puppet environment, install the prequequisite gems (hiera-eyaml, deep_merge, r10k) and populate the external modules directory via r10k, just run:
If you also want to install the recommended (Fabric, Vagrant, Docker) tools that can be used with the repo, run:
You will be asked to confirm or skip each component installation.
NOTE: Scripts are mostly tested on Mac and Linux environments, support on Windows is WIP.
For unattended setups (typically in CI pipelines) you can skip confirmation requests with:
The control-repo is full of more or less hidden stuff, which ease a lot Puppet code development, testing and deployment.
For more information on specific topics:
Development Workflow - An introduction of possible commands and workflows for Puppet code management
Vagrant Integration - How to use Vagrant to test the control-repo while deployment
Docker Integration - How to use Docker to test Puppet code and to build images based on the existing Puppet code
AWS Integration - How to use Puppet to query and configure AWS resources from the control-repo
Noop Mode - An overview on how to enforce noop mode server side with this repo
Trusted Facts - How to set and use trusted facts in this control-repo
Hiera eyaml - An overview on how to use hiera-eyaml
Git tasks - A review of Git tasks available with Fabric
Puppet tasks - A review of Puppet tasks available with Fabric
Tiny Puppet Integrations - Learn about the impressive things you can do with Tiny Puppet and this control-repo
example42 history - A summary of the evolution of example42 modules
Common tasks via Fabric
Many useful and common activities related to Puppet code development, testing and deployment can be fulfilled using Fabric.
Many Fabric tasks use scripts in the
bin/ directory. You can invoke them directly, if preferred.
Show available Fabric tasks (note: some will be run locally, some on the hosts specified via -H):
Run a Fabric task (better to do this from the main repo directory):
To have a full working environment you might need to locally install some software for specific activities.
You can simply run
bin/setup.sh to install them via Puppet or just can do that manually, as follows.
All the modules have a
metadata.json file where dependencies are described.
Most of the modules require PuppetLabs' stdlib.
Some modules (the ones, of generation 2.x, which use the
params_lookup function) require Puppi.
To be able to use the control-repo with Puppet some gems are needed and modules defined in the
Puppetfile have to be deployed.
The hiera-eyaml, r10k and deep_merge gems can be installed by the setup script or manually with commands like:
# Gem installation in system gem install hiera-eyaml gem install r10k gem install deep_merge # Gem installation in Puppet environment /opt/puppetlabs/puppet/bin/gem install hiera-eyaml /opt/puppetlabs/puppet/bin/gem install r10k /opt/puppetlabs/puppet/bin/gem install deep_merge # Gem installation in Puppet server environment (if present) /opt/puppetlabs/server/apps/puppetserver/cli/apps/gem install hiera-eyaml /opt/puppetlabs/server/apps/puppetserver/cli/apps/gem install r10k /opt/puppetlabs/server/apps/puppetserver/cli/apps/gem install deep_merge
Population of the
modules directory via r10k based on
r10k puppetfile install -v
The above steps can be accomplished by simply running
For a correct setup of the Vagrant environment you need Vagrant, VirtualBox and some extra plugins:
vagrant plugin install vagrant-cachier vagrant plugin install vagrant-vbguest vagrant plugin install vagrant-hostmanager
These plugins, as Vagrant itself, can be installed by the setup script.
Docker operations via Fabric or the command line require Docker to be locally installed.
If you use Mac or Windows you need the newer native client, things won't work when using Docker running inside a Virtualbox VM.
You'll need to run
docker login before trying any operation that involves pushing your images to Docker registry.
Also Docker can be installed by the setup script.
Using and understanding this control-repo
The control-repo you find here is what we consider a starting point for a state of the art general purpose Puppet setup.
It's based on a nodeless classification, driven by 3 top scope variables:
$::role- Defines the nodes' role
$::env- Defines the nodes' operational environment
$::zone- Defines the datacenter or region or segment of an infrastructure (optional)
These variables are used in the Hiera's hierarchy (check
docs/hiera.yaml) and should be enough to classify univocally any node in a averagely complex infrastructure. Here they are set as external facts (you'll need to set them when provisioning your nodes, as it's done in the Vagrant environment).
Such an approach can be easily adapted to any other logic and environment, for example, you can use an External Node Classifier (ENC) like Puppet Enterprise or The Foreman and manage there how your nodes are classified.
The manifests file,
manifests/site.pp sets some resource defaults, includes a baseline profile according to the underlying OS and uses hiera to define what profiles have to be included in each role (a more traditional alternative, based on role classes, is possible).
All the Hiera data is in
hieradata , the file
docs/hiera.yaml shows a possible hierarchy design and uses
hiera-eyaml as backend for keys encryption (no key is currently encrypted, because we are not shipping the generated private key (it's in .gitignore).
You will have to regenerate your hiera-eyaml keys (run, from the main repo dir,
On your Puppet server, if you want to keep hiera.yaml information in the control-repo you have to link it:
ln -sf /etc/puppetlabs/code/environments/production/docs/hiera.yaml /etc/puppetlabs/code/hiera.yaml
site directory there are local "not public" modules. Basically our profiles and some role examples.
For specifically there's the profile modules under
site/profile with a large amount of sample profiles for several common and not so sommon tasks.
There's also a tools module, under
site/tools which contains defines useful to manage common resources on a system.
modules directory are placed the public modules, as defined in the
Puppetfile and installed via r10k or librarian-puppet.
vagrant directory contains different Vagrant environments with the relevant toolset that can be used to test the same control-repo.
They are fully customizable by editing the
config.yaml file in each Vagrant environment.
Files for building Docker images locally are under the
skeleton directory contains a module skeleton you can use, and modify, to generate new modules based on the skeleton structure.
Documentation is stored under
docs, while the
bin directory contains several scirpts fot various purposes. Most of them can be invoked via Fabric, as configured in the
*.py files in the main directory.