Development repository for the consul cookbook
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
.delivery Use Chef Delivery CLI to run spec and style checks Apr 23, 2017
.github Update the contributing document. Apr 2, 2016
attributes Merge pull request #500 from visioncritical/stop_node_send Sep 3, 2018
libraries Merge pull request #500 from visioncritical/stop_node_send Sep 3, 2018
recipes Leverage nssm 4.0.0 and remove obsolete code Oct 1, 2017
templates/default Remove post-start stanza from upstart template May 27, 2017
test Fix message about what is being tested in spec test Jul 9, 2018
.foodcritic Fixed foodcritic with chef-14 Jul 9, 2018
.gitignore Update repository for release. Oct 12, 2017
.kitchen.dokken.yml Remove default attributes ['config']['owner'] and ['config']['group'] May 30, 2017
.kitchen.yml Merge branch 'master' into consul_v1 Jun 14, 2018
.rspec Update pending tests. Aug 23, 2016
.rubocop.yml Fix cookstyle offenses Jan 18, 2017
.travis.yml Merge branch 'master' into consul_v1 Jun 14, 2018
Berksfile Use Berkshelf for dep resolving in ChefSpec Apr 23, 2017
CHANGELOG.md Stop sending helper into node Jun 6, 2018
CONTRIBUTING.md added CONTRIBUTING.md and TESTING.md to improve the quality score on … May 26, 2018
Gemfile Fixed foodcritic with chef-14 Jul 9, 2018
LICENSE Update repository for release. Oct 12, 2017
Policyfile.rb Update testing suite to use policyfiles. Jul 10, 2016
README.md Update `go` version Nov 6, 2017
TESTING.md added CONTRIBUTING.md and TESTING.md to improve the quality score on … May 26, 2018
TODO.md Update repository for release. Oct 12, 2017
chefignore Initial commit creating the Consul cookbook. Apr 18, 2014
metadata.rb Ignore foodcritic for FC121 Jul 9, 2018

README.md

Consul Cookbook

Build Status Code Quality Cookbook Version License

Application cookbook which installs and configures Consul.

Consul is a tool for discovering and configuring services within your infrastructure. This is an application cookbook which takes a simplified approach to configuring and installing Consul. Additionally, it provides Chef primitives for more advanced configuration.

Basic Usage

For most infrastructure we suggest first starting with the default recipe. This installs and configures Consul from the latest supported release. It is also what is used to certify platform support through the use of our integration tests.

This cookbook provides node attributes which are used to fine tune the default recipe which installs and configures Consul. These values are passed directly into the Chef resource/providers which are exposed for more advanced configuration.

Out of the box the following platforms are certified to work and are tested using our Test Kitchen configuration. Additional platforms may work, but your mileage may vary.

  • RHEL/CentOS 5.11, 6.8, 7.3
  • Ubuntu 12.04, 14.04, 16.04
  • Debian 7.11, 8.7
  • Windows Server 2012 R2

Client

Out of the box the default recipe installs and configures the Consul agent to run as a service in client mode. The intent here is that your infrastructure already has a quorum of servers. In order to configure Consul to connect to your cluster you would supply an array of addresses for the Consul agent to join. This would be done in your wrapper cookbook:

node.default['consul']['config']['start_join'] = %w{c1.internal.corporate.com c2.internal.corporate.com c3.internal.corporate.com}

Server

This cookbook is designed to allow for the flexibility to bootstrap a new cluster. The best way to do this is through the use of a wrapper cookbook which tunes specific node attributes for a production server deployment.

The Consul cluster cookbook is provided as an example.

Advanced Usage

As explained above this cookbook provides Chef primitives in the form of resource/provider to further manage the install and configuration of Consul. These primitives are what is used in the default recipe, and should be used in your own wrapper cookbooks for more advanced configurations.

Configuration

It is very important to understand that each resource/provider has defaults for some properties. Any changes to a resource's default properties may need to be also changed in other resources. The best example is the Consul configuration directory.

In the example below we're going to change the configuration file from the default (/etc/consul.json) to one that may be on a special volume. It is obvious that we need to change the path where consul_config writes its file to, but it is less obvious that this needs to be passed into consul_service.

Inside of a recipe in your wrapper cookbook you'll want to do something like the following block of code. It uses the validated input from the configuration resource and passes it into the service resource. This ensures that we're using the same data.

config = consul_config '/data/consul/default.json'
consul_service 'consul' do
  config_file config.path
end

Security

The default recipe makes the Consul configuration writable by the consul service user to avoid breaking existing implementations. You can make this more secure by setting the node['consul']['config']['owner'] attribute to root, or set the owner property of consul_config explicitly:

# attributes file
default['consul']['config']['owner'] = 'root'

or

# recipe file
consul_config '/etc/consul/consul.json' do
  owner 'root'
end

Watches/Definitions

In order to provide an idempotent implementation of Consul watches and definitions. We write these out as a separate configuration file in the JSON file format. The provider for both of these resources are identical in functionality.

Below is an example of writing a Consul service definition for the master instance of Redis. We pass in several parameters and tell the resource to notify the proper instance of the Consul service to reload.

consul_definition 'redis' do
  type 'service'
  parameters(tags: %w{master}, address: '127.0.0.1', port: 6379)
  notifies :reload, 'consul_service[consul]', :delayed
end

A check definition can easily be added as well. You simply have to change the type and pass in the correct parameters. The definition below checks memory utilization using a script on a ten second interval.

consul_definition 'mem-util' do
  type 'check'
  parameters(script: '/usr/local/bin/check_mem.py', interval: '10s')
  notifies :reload, 'consul_service[consul]', :delayed
end

A service definition with an integrated check can also be created. You will have to define a regular service and then add a check as a an additional parameter. The definition below checks if the vault service is healthy on a 10 second interval and 5 second timeout.

consul_definition 'vault' do
  type 'service'
  parameters(
    port:  8200,
    address: '127.0.0.1',
    tags: ['vault', 'http'],
    check: {
      interval: '10s',
      timeout: '5s',
      http: 'http://127.0.0.1:8200/v1/sys/health'
    }
  )
  notifies :reload, 'consul_service[consul]', :delayed
end

Finally, a watch is created below to tell the agent to monitor to see if an application has been deployed. Once that application is deployed a script is run locally. This can be used, for example, as a lazy way to clear a HTTP disk cache.

consul_watch 'app-deploy' do
  type 'event'
  parameters(handler: '/usr/local/bin/clear-disk-cache.sh')
  notifies :reload, 'consul_service[consul]', :delayed
end

A keen eye would notice that we are delaying the reload of the Consul service instance. The reason we do this is to minimize the number of times we need to tell Consul to actually reload configurations. If there are several definitions this may save a little time off your Chef run.

ACLs

The consul_acl resource allows management of Consul ACL rules. Supported actions are :create and :delete. The :create action will update/insert as necessary.

The consul_acl resource requires the Diplomat Ruby API gem to be installed and available to Chef before using the resource. This can be accomplished by including consul::client_gem recipe in your run list.

In order to make the resource idempotent and only notify when necessary, the id field is always required (defaults to the name of the resource). If type is not provided, it will default to "client". The acl_name and rules attributes are also optional; if not included they will be empty in the resulting ACL.

The example below will create a client ACL token with an ID of the given UUID, Name of "AwesomeApp Token", and Rules of the given string.

consul_acl '49f06aa9-782f-465a-becf-44f0aaefd335' do
  acl_name 'AwesomeApp Token'
  type 'client'
  rules <<-EOS.gsub(/^\s{4}/, '')
    key "" {
      policy = "read"
    }
    service "" {
      policy = "write"
    }
  EOS
  auth_token node['consul']['config']['acl_master_token']
end

Execute

The command-line agent provides a mechanism to facilitate remote execution. For example, this can be used to run the uptime command across your fleet of nodes which are hosting a particular API service.

consul_execute 'uptime' do
  options(service: 'api')
end

Warning on git based installs

Consul v1.0 states that Go 1.9 is a requirement. The default go installation uses 1.5, so you may need to override a ['go']['version'] attribute to allow the git installation to work reliably.

All of the options available on the command-line can be passed into the resource. This could potentially be a very dangerous operation. You should absolutely understand what you are doing. By the nature of this command it is impossible for it to be idempotent.