Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

INSTALL

jfleong edited this page · 100 revisions

Ironfan Installation Instructions

Conventions

In all of the below,

  • {homebase}: is the directory that holds your Chef cookbooks, roles and so forth. For example, this file is in {homebase}/README.md.
  • {username}: identifies your personal Chef client name: the thing you use to log into the Chef WebUI.
  • {organization}: identifies the credentials set and cloud settings to use. If your Chef server is on the Opscode platform (Try it! It's super-easy), use your organization name (the last segment of your chef_server url). If not, use an identifier you deem sensible.

Cloud Accounts

This installation presumes that you have several hosted service accounts. All of them have a free tiers: see Amazon Web Services, Hosted Chef from Opscode, and Github for more details and to sign up.

Install platform-specific prerequisites

The goal of this is install Ruby v1.9.3, and any platform prerequisites for the bundled gems.

Macintosh

  1. Install "Command Line Tools for XCode" from https://developer.apple.com/xcode/
  2. Install homebrew

    # From instructions at http://mxcl.github.com/homebrew/
    ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
    brew update
    
  3. Install rbenv and ruby build

    brew install rbenv
    brew install ruby-build
    echo 'eval "$(rbenv init -)"' >> ~/.bash_profile
    source ~/.bash_profile
    

note: (or .zshrc for a bunch of us)

  1. Build and use ruby 1.9.3 rbenv install list # to get the current release number rbenv install 1.9.3-{current release} # current 1.9.3-p550 2014-10-29 rbenv global 1.9.3-{current release} # or rbenv local in {homebase}

Debian/Ubuntu

(Note: Ubuntu 13.10 has native ruby 1.9, and can do sudo apt-get install ruby libruby ruby-bundler libxml2-dev libxslt1-dev instead of the rbenv approach listed here. This does put bundler in charge of your system-wide gems directory, unless you do all your bundle install calls with --path=~/.bundle.)

  1. Install gem native dependencies

    sudo apt-get install git libxml2-dev libxslt1-dev libreadline-dev zlib1g-dev
    
  2. Install rbenv and ruby-build

    git clone git://github.com/sstephenson/rbenv.git ~/.rbenv
    echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bash_profile
    
    git clone git://github.com/sstephenson/ruby-build.git ~/.rbenv/plugins/ruby-build
    echo 'eval "$(rbenv init -)"' >> ~/.bash_profile
    
    source ~/.bash_profile
    
  3. Build and use ruby 1.9

    sudo apt-get remove clang    # This conflicts with the ruby build process
    rbenv install list # to get the current release number
    rbenv install 1.9.3-{current release} # current 1.9.3-p550 2014-10-29
    rbenv global 1.9.3-{current release}       # or rbenv local in {homebase}
    

Windows

This is necessary for a POSIX command line, even if you already have Ruby.

  1. Install Cygwin with the gcc, git, libxml2-devel, libxslt-devel, make, openssh, and ruby packages.
  2. Run (Windows-R) C:/cygwin/bin/ash /usr/bin/rebaseall

FreeBSD

From the command line

    sudo pkg install -r libxslt readline git ruby19 ruby-gems

This should work for 6.4 through 9.1. For 10.0, you may have to compile one or more packages manually, due to frozen dependencies.

Install the bundler gem

From the command line (Terminal, Cygwin, etc.)

    gem install bundler

(You may need to prefix this with sudo for it to work correctly. If you are using rbenv, you should follow it with a rbenv rehash.)

Install Ironfan's Homebase

A homebase is the place where cookbooks, roles, config files and other artifacts for managing systems with Chef will live.

  1. Fork the homebase repo on Github, as you'll be making changes to personalize it for your platform that you may want to share with teammates.
  2. Clone the repo (making sure to substitute the correct value for organization):

    git clone git@github.com:{organization}/ironfan-homebase.git
    # or git clone https://github.com/{organization}/ironfan-homebase
    cd ironfan-homebase
    export IRONFAN_HOMEBASE=`pwd`
    bundle install
    git submodule update --init
    

Troubleshooting: if you see the following error:

    Building nokogiri using packaged libraries.
    -----
    libiconv is missing.  please visit http://nokogiri.org/tutorials/installing_nokogiri.html for help with installing dependencies.

You need to solve nokogiri depedencies most likely by: OSX (doc):

    xcode-select --install 

Configure Knife and Add Credentials

Ironfan expands out the traditional singular knife.rb into several components. This modularity allows for better management of sensitive shared credentials, personal credentials, and organization-wide configuration.

Note: If your local username differs from your Opscode Chef username, run the following (adding the correct Chef user name):

    echo 'export CHEF_USER={username}' >> ~/.bash_profile
    source ~/.bash_profile

Credentials Directory

All the keys and settings specific to your organization are held in a directory named credentials/, versioned independently of the homebase.

To set up your credentials directory:

    cd $IRONFAN_HOMEBASE/knife 
    cp -a credentials local-credentials

You will likely want to store the credentials in another remote repository. We recommend erring on the side of caution in its hosting. Setting that up is outside the scope of this guide, but there good external resources available to get you started.

Download Cloud Credentials

You will need to obtain user keys from your cloud providers. Your AWS access keys can be obtained from Amazon IAM:

Reset AWS User Key


Your Opscode user key can be obtained from the Opscode Password settings console:

Reset Opscode User Key

(Self-hosted Chef servers have a different security setup. You will need to create a client with the same name as the user, give it sufficient permissions, and download that client's PEM file, instead.)

User / Organization-specific config

Edit the following in your new credentials:

  • Organization-specific settings are in knife/credentials/knife-org.rb:

    • organization: Your organization name
    • chef server url: Edit the line for your chef_server_url. Note: If you are an Opscode platform user, you can skip this step -- your chef_server_url defaults to https://api.opscode.com/organizations/#{organization}.
    • Cloud-specific settings: if you are targeting a cloud provider, add account information and configuration here.
  • User-specific settings are in knife/credentials/knife-user-{username}.rb. (You can duplicate and rename the one in knife/example-credentials/knife-user-example.rb). For example, if you're using Amazon EC2 you should set your access keys:

      knife[:aws_access_key_id]      = "XXXX"
      knife[:aws_secret_access_key]  = "XXXX"
    
  • Put the Chef user key in knife/credentials/{username}.pem at.

  • If you have existing Amazon machines, place their keypairs in knife/credentials/ec2_keys. Ironfan will also automatically populate this with new keys as new clusters are created.

Test access

    bundle exec knife client list

...should run without error and list a couple of clients configured at the server.

Fetch cookbooks, and upload all cookbooks/roles/environments to chef server

    bundle exec rake full_sync

Try it out

Copy some example clusters into your cluster folder

    cp example_clusters/*.rb clusters/

You should now be able to use Knife to control your clusters:

    $ bundle exec knife cluster list
    +--------------------+---------------------------------------------------+ 
    | cluster            | path                                              |
    +--------------------+---------------------------------------------------+
    | burninator         | /cloud/clusters/burninator.rb                     |
    | el_ridiculoso      | /cloud/clusters/el_ridiculoso.rb                  |
    | elasticsearch_demo | /cloud/clusters/elasticsearch_demo.rb             |
    | hadoop_demo        | /cloud/clusters/hadoop_demo.rb                    |
    | sandbox            | /cloud/clusters/sandbox.rb                        |
    +--------------------+---------------------------------------------------+

Launching a cluster in the cloud should now be this easy!

    bundle exec knife cluster launch sandbox-simple

Next

The README file in each of the subdirectories for more information about what goes in those directories. If you are bored of reading, go customize one of the files in the 'clusters/ directory'. Or, if you're a fan of ridiculous things and have ever pondered how many things you can fit in one box, launch el_ridiculoso:. It contains every single recipe we have ever made stacked on top of one another.

    knife cluster launch el_ridiculoso-gordo --bootstrap

For more information about configuring Knife, see the Knife documentation.

Something went wrong with that request. Please try again.