octocatalog-diff is a tool that enables developers to be more efficient when testing changes to Puppet manifests. It is most commonly used to display differences in Puppet catalogs between stable and development branches. It does not require a working Puppet master (or puppetserver), so it is often run by developers on their workstations and in Continuous Integration environments.
At GitHub, we manage thousands of nodes with a Puppet code base containing 500,000+ lines of code from over 200 contributors. We run
octocatalog-diff thousands of times per day as part of Continuous Integration testing, and developers run it on their workstations as they are working with the code.
octocatalog-diff is written in Ruby and is distributed as a gem. It runs on Mac OS and Unix/Linux platforms.
The current version of
octocatalog-diff in the master branch is 0.99.rc1, a release candidate for version 1.0.
We consider the 1.x release of
octocatalog-diff to be stable and production-quality. We continue to maintain and enhance
octocatalog-diff to meet GitHub's internal needs and to incorporate suggestions from the community. Please consult the change log for details.
Traditional Puppet development generally takes one of two forms. Frequently, developers will test changes by running a Puppet agent (perhaps in
--noop mode) to see if the desired change has resulted on an actual system. Others will use formal testing methodologies, such as
rspec-puppet or the beaker framework to validate Puppet code.
octocatalog-diff uses a different pattern. In its most common invocation, it compiles Puppet catalogs for both the stable branch (e.g. master) and the development branch, and then compares them. It filters out attributes or resources that have no effect on ultimate state of the target system (e.g. tags) and displays the remaining differences. Using this strategy, one can get feedback on changes without deploying Puppet code to a server and conducting a full Puppet run, and this tool works even if test coverage is incomplete.
There are some limitations to a catalog-based approach, meaning it will never completely replace unit, integration, or deployment testing. However, it does provide substantial time savings in both the development and testing cycle. In this repository, we provide example scripts for using
octocatalog-diff in development and CI environments.
octocatalog-diff is currently able to get catalogs by the following methods:
- Compile catalog via the command line with a Puppet agent on your machine (as GitHub uses the tool internally)
- Obtain catalog over the network from PuppetDB
- Obtain catalog over the network using the API to query a Puppet Master / PuppetServer (Puppet 3.x and 4.x supported)
- Read catalog from a JSON file
Here is simulated output from running
octocatalog-diff to compare the Puppet catalog changes between the master branch and the Puppet code in the current working directory:
The example above reflects the changes in the Puppet catalog from switching an underlying device for a mounted file system.
Installation and use in a development environment
Installation and use for CI
What's in a name?
During its original development at GitHub, this tool was simply called
catalog-diff. However, there is already a Puppet module with that name and we didn't want to create any confusion (in fact, a case could be made to use both approaches). So, we named the tool
octocatalog-diff because who doesn't like the octocat? Then one day in chat, someone referred to the tool as "alog-diff", and that moniker caught on for electronic communication.
Please see our contributing document if you would like to participate!
octocatalog-diff is licensed under the MIT license.
It requires 3rd party ruby gems found here. It also includes portions of other open source projects here, here, here and here. All 3rd party code and required gems are licensed either as MIT or Apache 2.0.
octocatalog-diff was designed and authored by Kevin Paulisse and is now maintained, reviewed, and tested by Kevin and the rest of the Site Reliability Engineering team at GitHub.