Leveraging Chef-Solo for Less than Enterprise Scale Deployments
Started at HMS: "here's a screw driver, your desk is in that box, there are your 2 computers here are the install disks" Up and running later that afternoon...the other guy not so much... ]] developed our own cluster of linux boxes all by hand, perl based system >> started some tools: fast-cluster-run parallel-jobs the 'm' engine >> all ssh / non-blocking IO based >> did the job My use of Chef started when I was at Algorithmics: joined a team of 2 as the 3rd person. single dev machine set up "Uh, here's the ubuntu ISO, and I think you need Rails, RabbitMQ, and ..." At this point my personality balks at repetetive tasks. I'm alergic, honest I am! :) Chef and Puppet seemed too complex for a situation where I thought I could just could write a few lines of bash (who hasn't?) So I started a script that automated the install - this served to: document the stack and software (all the required packages and steps were listed in the script) fix versions of most of our dependencies (provided stability) it automated the set up process Process took me about a day to get going Few weeks later had to do a 3rd workstation. Took ~2hrs => win! Got Virtual Hosting Slices, <2hs to set each one up (win!) Demo Time: Fast Fwd to Oct 2009, Hotel Ballroom in Manhattan (NYC) Oops! provider has issues 2 days before. Gulp! 9am decision is made to procure backup box with another hosting provider, get additional domain name, SSL cert, etc. ~1pm, full stack is installed, tested, up and running Demo goes off w/o too many hitches. In Jan/Feb of 2010 timeframe we started migrating the hokey bash script to Chef. I joined a startup in May of this year. Decided to start with Chef right off the bat. No more bash, use a real tool.
In Retail it’s “Location! Location! Location!”
In Systems it’s “Automation! Automation! Automation!” (at least it is for me)
automation saves time / resources configuration management: consistency, known states, record of changes easy to add new roles, workstations, servers as environments change, as we need capacity
Where I’ve used Chef:
Snapclean.me: 1 box, handful of recipies for provisioning & system admin, not for app deployment (using capistrano for that) -- describe the difference I see between the two types of tools and why Relay: 8 boxes @ softlayer; dev workstation; testing/staging/CI/CD environment at Linode multiple roles multiple environments
So what is Chef? Solo vs Server?
What’s in a cookbook?
Attributes, the recipes
Resources: Packages, Scripts, Files, Templates
Nodes and Roles
You still need to have discipline. Seems to be a trend to define this as a role: devops. Some feel 'developers' should not have access to production. Some feel tasks are better left to system administrators. Don't fall into the trap of over engineering! Keep it lean, manage the provisioning like any other feature set in your organization, stay precient of requiremetns, benefits - the ROI, don't get overly complex initially. Practice, Practice, Practice: use Linode/Slicehost accounts (cheap), use virtualbox, use vmware. Chef (and Puppet) are DIY toolkits / frameworks, you will have to write / maintain your systems - but it'll have all those advantages (config mgmt, automation, consistency) To me these seem like 2nd generation system administration 1st gen: manual + adhoc scripting; single-system automation, not cross-platform (upstart, etc) 2nd gen: rise of automation frameworks: standardization of techniques here b/c of the rise of cloud computing - fungible computing resources, starting to support heterogenus coputer resources (different OS flavors) 3rd gen: still in the future IMO, software will move toward being aware of being commissioned / decomossioned (i.e. used on a dynamic cloud); we're seeing some of this, but it's not prevelent yet IMO; operating systems (esp linux and the bsd's) will begin to move more toward automation These kinds of dynamic systems do exist, but are largely in advanced 'enterprises' or large companies that have the staff / expertiese to do it. It's expensive as heck - Chef/Puppet are a sign (to me) that the cost of doing this is reducing and people are finding ways of leveraging larger scales of compute resources with smaller teams and budgets and doing interesting things on less margin Chef is a great force for automaiton, you can use it to keep discipline you can (and should) use it to scale your resources further; making your dev's responsible for helping to ensure production is continuously availble will lead to chanes in your organization - for the better
What is Chef?
I wish I had more experience with Chef. I’m able to use it, but I know I’m not
leveraging all of it’s power. You’re seeing me part way through my journey
with this tool.
Where / When is it appropriate?
Chef is an automation framework. It is not a menu of node types to choose from.
Nodes, Cookbooks, Recipies, Resources.
Chef decouples the idea of a role (node) from a physical box. I create a node,
‘app server’, but I run it on multiple boxes.
Chef is declarative. You write recipes that declare the tasks that need to be
executed. Your recipes leverage a combination of the node’s configuration
(json or ruby), and what you detect on the machine you’re doing the install on.
Need to validate: if 2 resources have the same exact ‘name’ (description), only
the first one ‘registered’ will actually execute.
Tooth Brushes (Good Habits)