Fog driver for Chef Provisioning
Ruby Shell
tas50 Merge pull request #227 from chef/testing_update
Update travis config for latest ruby releases
Latest commit 6855e39 Jul 26, 2018
Failed to load latest commit information.
examples Merge pull request #197 from NCR-Corporation/vcair-changes Mar 3, 2017
lib/chef bump version for release Apr 24, 2018
spec Add provider for Scaleway Dec 10, 2016
support Improve handling of fog credentials Feb 19, 2016
.travis.yml Update travis config for latest ruby releases Jul 26, 2018 Update the changelog Nov 27, 2017
Gemfile remove direct fog dep and include meta-deps directly Oct 19, 2017
Guardfile set up rspec Jun 11, 2014
LICENSE Move Fog driver here from chef-metal Apr 4, 2014 update Oct 23, 2017
Rakefile update for latest gems Sep 28, 2017


Gem Version Build Status Dependency Status

This is the Fog driver for Chef Provisioning. It provides Amazon EC2, DigitalOcean, Google Compute Engine, IBM Softlayer, Joyent, OpenStack, Rackspace, vCloud Air and XenServer functionality.


These are the primary documents to help learn about using Provisioning and creating Provisioning drivers:

chef-provisioning-fog Usage and Examples

A note about key pairs - The key name used in fog_key_pair must be the same as the filename of the local key to be used. If the key does not exist in one of private_key_paths (which you can set in knife.rb or in a client.rb) it will be created.

Amazon EC2

NOTE: Using chef-provisioning-fog for AWS is discouraged and new users should use chef-provisioning-aws. To use chef-provisioning-fog the fog-aws gem must be installed by hand (it is not list a dependency of this gem any more). ChefDK users should chef gem install fog-aws.

You'll need to update your knife.rb to work:

You need one of the following for the driver:


For example:

driver 'fog:AWS:<account_id>:<region>'
driver_options :compute_options => {
                                     :aws_access_key_id => 'YOUR-ACCESS-KEY-ID',
                                     :aws_secret_access_key => 'YOUR-SECRET-ACCESS-KEY',
                                     :region => 'THE-REGION-YOU-WANT-TO-PROVISION-IN'

For a full example see examples/aws/simple.rb.


Update your knife.rb to contain your DigitalOcean API token and the driver:

driver 'fog:DigitalOcean'
driver_options :compute_options => {
                                    :digitalocean_token => 'token'

fog 1.38.0 (Newest release) has an issue with DigitalOcean. If you want to use DigitalOcean and chef-provisioning-fog you need to down grade your fog gem.

$ gem install fog --version 1.37.0
Successfully installed fog-1.37.0
Parsing documentation for fog-1.37.0
$ gem uninstall fog

Select gem to uninstall:
 1. fog-1.37.0
 2. fog-1.38.0
 3. All versions
> 2
Successfully uninstalled fog-1.38.0

For a full example see examples/digitalocean/simple.rb.

Google Compute Engine

NOTE: currently fog-google is broken against the current google-api-client client. The dependency on fog-google has been removed until this is fixed upstream. Users will likely need to use a Gemfile and pin manually to a working version of fog, fog-google and google-api-client. It is unlikely that it will work successfully with ChefDK with no Gemfile

You'll need to update your knife.rb to work:

driver 'fog:Google'
driver_options :compute_options => {
                                     :provider => 'google',
                                     :google_project => 'YOUR-PROJECT-ID', # the name will work here
                                     :google_client_email => 'YOUR-SERVICE-ACCOUNT-EMAIL',
                                     :google_key_location => 'YOUR-SERVICE-P12-KEY-FILE-FULL-PATH.p12'

In order to get the YOUR-SERVICE-P12-KEY-FILE.p12 you need to set up a Service account. This is located at Home > Permissions > Service Accounts and you'll need to create a new one to get a new key. After that place it some place such as ~/.chef/ so chef-provisioning-fog can find it. Your google_client_email would be something like: <UNIQUE_NAME>@<PROJECT>

For a full simple example see examples/google/simple.rb.

For an example that shows a different :disk_type see examples/google/simple_different_disk.rb.

IBM SoftLayer

You'll need to update your knife.rb to work with this also:

driver 'fog:SoftLayer'
driver_options :compute_options => {
                                     :provider => 'softlayer',
                                     :softlayer_username => 'username',
                                     :softlayer_api_key => 'api_key',
                                     :softlayer_default_domain => '',

Once you or your administrator has created a SoftLayer account you can manage your API key at

:bootstrap_options => {:key_name => 'label'} is looked up by_label; make sure you have a public key created on control portal at with a matching label.

NOTE: the SoftLayer driver injects a custom post provisioning script that ensures some packages needed by chef-provisioning-fog to install chef are present (e.g. sudo). The injected script will call your :postInstallScriptUri if you define one. The driver will wait until the injected script is done. The driver and script communicate using userMetadata so you cannot use metadata.

For a full example see examples/softlayer/simple.rb.


You'll need to update your knife.rb to work:

driver 'fog:Joyent'
driver_options :compute_options => {
                                     :joyent_username => 'YOUR-JOYENT-LOGIN',
                                     :joyent_password => 'YOUR-JOYENT-PASSWORD',
                                     :joyent_keyname => 'THE-SSH-KEY-YOUVE-UPLOADED',
                                     :joyent_version => '7.3.0', # if you are using the joyent public cloud
                                     :joyent_keyfile => 'YOUR-PRIVATE-SSH-KEY-LOCATION' # Such as '/Users/jasghar/.ssh/id_rsa'

Tested this with the Joyent Public Cloud. For the package names, use the GUI to find the name(s) that you want to use. This is also required to figure out the Image UUID, there doesn't seem to be an effective way of doing this without the GUI.

For a more in-depth usage of this driver to use with either Private or Public Joyent cloud, checkout this blog post by mhicks from #smartos on freenode.

For a infrastructure container example see examples/joyent/infrastructure.rb.


You'll need to update your knife.rb to work:

driver 'fog:OpenStack'
driver_options :compute_options => {
                                     :openstack_auth_url => 'http://YOUROPENSTACK-CLOUD:5000/v2.0/tokens',
                                     :openstack_username => 'YOUR-USERNAME',
                                     :openstack_api_key  => 'YOUR-PASSWORD',
                                     :openstack_tenant   => 'YOUR-TENANT-ID-NAME'

For a full example see examples/openstack/simple.rb.


For this example, you must configure knife.rb with your credentials and a region to operate on. This example is also available as a Github repo.

You must configure some credentials and region in a knife.rb file like so:

driver 'fog:Rackspace'
driver_options :compute_options => {
                                     :rackspace_username => 'MY-RACKSPACE-USERr',
                                     :rackspace_api_key  => 'API-KEY-FOR-USER',
                                     :rackspace_region => 'dfw' # could be 'org', 'iad', 'hkg', etc

For a full example see examples/rackspace/simple.rb.

vCloud Air

NOTE: The fog mega-gem has been removed as a direct dependency of chef-provisioning-fog. Since support for vcair is only in the fog gem and fog does not supply any "meta-gem" for vcair specifically, that means that the fog gem must be manually installed. ChefDK users should manually chef gem install fog.

Docs TODO.


You should configure XenServer driver with your credentials:

with_driver "fog:XenServer:<XEN-SERVER-IP/NAME>", compute_options: {
  xenserver_username: 'MY-XEN-USER',
  xenserver_password: 'MY-XEN-PASSWORD',
  xenserver_defaults: {
    'template' => 'ubuntu-14.04'

For a full example see examples/xenserver/simple.rb.


You should configure the driver with your credentials or :

with_driver "fog:Scaleway:Your-Organisation-UUID:region", compute_options: {
  scalewat_token: 'your-api-token',

or just use the fog configuration (~/.fog):

with_driver 'fog:Scaleway'

For full examples, see examples/scaleway.

Cleaning up

require 'chef/provisioning'

machine_batch do
  machines search(:node, '*:*').map { |n| }
  action :destroy

When you are done with the examples, run this to clean up:

$ chef-client -z destroy_all.rb

What Is Chef Provisioning?

Chef Provisioning has two major abstractions: the machine resource, and drivers.

The machine resource

You declare what your machines do (recipes, tags, etc.) with the machine resource, the fundamental unit of Chef Provisioning. You will typically declare machine resources in a separate, OS/provisioning-independent file that declares the topology of your app--your machines and the recipes that will run on them.

The machine resources from the cluster.rb example are pretty straightforward. Here's a copy/paste, it'll create a database machine then one web server.

machine 'mario' do
  recipe 'postgresql'
  recipe 'mydb'
  tag 'mydb_master'

num_webservers = 1

1.upto(num_webservers) do |i|
  machine "luigi#{i}" do
    recipe 'apache'
    recipe 'mywebapp'

You will notice the dynamic nature of the number of web servers. It's all code, your imagination is the limit :)


Drivers handle the real work of getting those abstract definitions into real, physical form. They handle the following tasks, idempotently (you can run the resource again and again and it will only create the machine once--though it may notice things are wrong and fix them!):

  • Acquiring machines from the cloud, creating containers or VMs, or grabbing bare metal
  • Connecting to those machines via ssh, winrm, or other transports
  • Bootstrapping chef onto the machines and converging the recipes you suggested

The driver API is separated out so that new drivers can be made with minimal effort (without having to rewrite ssh, tunneling, bootstrapping, and OS support). But to the user, they appear as a single thing, so that the machine acquisition can use its smarts to autodetect the other bits (transports, OS's, etc.).

Drivers save their data in the Chef node itself, so that they will be accessible to everyone who is using the Chef server to manage the nodes.

Tuning Timeouts

chef-provisioning-fog interacts with your cloud provider's endpoint. Most of the time, default timeouts for the following would be sufficient.

Fog connection_options

Modify the defaults if your Fog endpoint takes awhile to send/receive API requests. Normally, if you see Excon timeouts you should tune these parameters.

with_driver 'fog:foo',
  :compute_options => {
    :connection_options => {
      # set connection to persist (default is false)
      :persistent => true,
      # set longer connect_timeout (default is 60 seconds)
      :connect_timeout => 360,
      # set longer read_timeout (default is 60 seconds)
      :read_timeout => 360,
      # set longer write_timeout (default is 60 seconds)
      :write_timeout => 360,

machine_option timeouts

Modify these timeouts if you need Chef to wait a bit of time to allow for the machine to be ready.

  # set longer to wait for the instance to boot to ssh (defaults to 180)
  :create_timeout => 360,
  # set longer to wait for the instance to start (defaults to 180)
  :start_timeout => 360,
  # set longer to wait for ssh to be available if the instance is detected up (defaults to 20)
  :ssh_timeout => 360

Chef Client convergence_options

Modify this if your chef client convergences take awhile.

  :convergence_options => {
    # set longer if you need more time to converge (default: 2 hours)
    :chef_client_timeout => 120*60