Leveraging Chef-Solo for Less than Enterprise Scale Deployments
Switch branches/tags
Nothing to show
Clone or download
Latest commit 8894ef9 Nov 10, 2010

README.textile

Leveraging Chef-Solo for Less than Enterprise Scale Deployments

Some context:

      Started at HMS: "here's a screw driver, your desk is in that box, there are
                       your 2 computers here are the install disks"
    
          Up and running later that afternoon...the other guy not so much...

        ]] developed our own cluster of linux boxes all by hand, perl based system
        >> started some tools: fast-cluster-run parallel-jobs the 'm' engine
          >> all ssh / non-blocking IO based
          >> did the job
    
      My use of Chef started when I was at Algorithmics:
    
         joined a team of 2 as the 3rd person.
    
         single dev machine set up
    
         "Uh, here's the ubuntu ISO, and I think you need Rails, RabbitMQ, and ..."

      At this point my personality balks at repetetive tasks.  I'm alergic, honest I am! :)

      Chef and Puppet seemed too complex for a situation where I thought I
      could just could write a few lines of bash (who hasn't?) So I started a
      script that automated the install - this served to:

         document the stack and software (all the required packages and steps were listed in the script)
         fix versions of most of our dependencies (provided stability)
         it automated the set up process

      Process took me about a day to get going

      Few weeks later had to do a 3rd workstation.  Took ~2hrs => win!

      Got Virtual Hosting Slices, <2hs to set each one up (win!)

      Demo Time: Fast Fwd to Oct 2009, Hotel Ballroom in Manhattan (NYC)

      Oops! provider has issues 2 days before.  Gulp!

      9am decision is made to procure backup box with another hosting provider,
      get additional domain name, SSL cert, etc.

      ~1pm, full stack is installed, tested, up and running

      Demo goes off w/o too many hitches.
      
    In Jan/Feb of 2010 timeframe we started migrating the hokey bash script to Chef.

    I joined a startup in May of this year.  Decided to start with Chef right
    off the bat.  No more bash, use a real tool.

In Retail it’s “Location! Location! Location!”

In Systems it’s “Automation! Automation! Automation!” (at least it is for me)

    automation saves time / resources

    configuration management: consistency, known states, record of changes

    easy to add new roles, workstations, servers as environments change, as we need capacity

Where I’ve used Chef:

   Snapclean.me: 1 box, handful of recipies
     for provisioning & system admin, not for app deployment (using capistrano for that)
      -- describe the difference I see between the two types of tools and why

   Relay: 8 boxes @ softlayer; dev workstation; testing/staging/CI/CD environment at Linode
      multiple roles
      multiple environments

So what is Chef? Solo vs Server?

Cookbooks, recipes

What’s in a cookbook?

Attributes, the recipes

Resources: Packages, Scripts, Files, Templates

Nodes and Roles

Conclusions:

    You still need to have discipline.

    Seems to be a trend to define this as a role: devops.

    Some feel 'developers' should not have access to production.  Some feel tasks
    are better left to system administrators.

    Don't fall into the trap of over engineering!  Keep it lean, manage the
    provisioning like any other feature set in your organization, stay precient of
    requiremetns, benefits - the ROI, don't get overly complex initially.

    Practice, Practice, Practice: use Linode/Slicehost accounts (cheap), use
    virtualbox, use vmware.
   

    Chef (and Puppet) are DIY toolkits / frameworks, you will have
    to write / maintain your systems - but it'll have all those advantages
    (config mgmt, automation, consistency)

      To me these seem like 2nd generation system administration 
        1st gen: manual + adhoc scripting; single-system automation, not
          cross-platform (upstart, etc)
        2nd gen: rise of automation frameworks: standardization of techniques
          here b/c of the rise of cloud computing - fungible computing resources,
          starting to support heterogenus coputer resources (different OS flavors)
        3rd gen: still in the future IMO, software will move toward being aware
          of being commissioned / decomossioned (i.e. used on a dynamic cloud);
          we're seeing some of this, but it's not prevelent yet IMO; operating
          systems (esp linux and the bsd's) will begin to move more toward automation

          These kinds of dynamic systems do exist, but are largely in advanced
          'enterprises' or large companies that have the staff / expertiese to
          do it.  It's expensive as heck - Chef/Puppet are a sign (to me) that
          the cost of doing this is reducing and people are finding ways of
          leveraging larger scales of compute resources with smaller teams
          and budgets and doing interesting things on less margin

    Chef is a great force for automaiton, you can use it to keep discipline
    you can (and should) use it to scale your resources further; making
    your dev's responsible for helping to ensure production is continuously availble
    will lead to chanes in your organization - for the better

What is Chef?

I wish I had more experience with Chef. I’m able to use it, but I know I’m not
leveraging all of it’s power. You’re seeing me part way through my journey
with this tool.

Where / When is it appropriate?

Chef is an automation framework. It is not a menu of node types to choose from.

Nodes, Cookbooks, Recipies, Resources.

Chef decouples the idea of a role (node) from a physical box. I create a node,
‘app server’, but I run it on multiple boxes.

Chef is declarative. You write recipes that declare the tasks that need to be
executed. Your recipes leverage a combination of the node’s configuration
(json or ruby), and what you detect on the machine you’re doing the install on.

Need to validate: if 2 resources have the same exact ‘name’ (description), only
the first one ‘registered’ will actually execute.

References:

Images

Records

Tooth Brushes (Good Habits)

Log Book / Ledger

Training Wheels