Permalink
Browse files

Merge git://github.com/tobami/littlechef

  • Loading branch information...
2 parents 3c2414b + 28aaa3a commit d8f14f855f0ac60c6a288cdd874b840b796f1e13 @VanL committed Jun 5, 2011
Showing with 150 additions and 70 deletions.
  1. +5 −4 CHANGELOG
  2. +1 −1 MANIFEST.in
  3. +13 −11 README.md
  4. +26 −11 littlechef/chef.py
  5. +45 −0 littlechef/data_bags_patch.rb
  6. +7 −2 littlechef/littlechef.py
  7. +11 −7 littlechef/solo.py
  8. +30 −17 littlechef/tests.py
  9. +1 −1 littlechef/version.py
  10. +4 −3 setup.py
  11. +0 −10 tests.py
  12. +7 −3 tests/tests.py
View
9 CHANGELOG
@@ -1,15 +1,16 @@
-======================
+====================
LittleChef Changelog
-======================
+====================
-Version 0.6.0 , 2011
+Version 0.6.0 June 3, 2011
----------------------------------------
+* NEW: Added support for data bags thanks to Brian Akins's Chef Solo patch
* NEW: Added option to install Chef 0.10 from debian packages. Not yet default
* NEW: Upgraded to use rubygems 1.7.2 in the gem installation
* NEW: Added support for deploying Chef on debian wheezy (testing)
* NEW: aVenger changed the behaviour of the commands 'recipe' and 'role' so that
- they preserve attributes of node which have a configuration file
+ they preserve attributes of nodes which already have a configuration file
Version 0.5.5 May 16, 2011
View
2 MANIFEST.in
@@ -1,3 +1,3 @@
-include README.textile
+include README.md
include LICENSE
include NOTICE
View
24 README.md
@@ -17,10 +17,11 @@ It all starts in the **kitchen**, which you should keep under version control:
* `cookbooks/`: This will be your [Cookbooks][] repository
* `site-cookbooks/`: Here you can override upstream cookbooks (Opscode's, for example)
* `roles/`: Where Chef [Roles][] are defined
+* `data_bags/`: Chef [Data Bags][]
-Whenever you apply a recipe to a node, all needed cookbooks (including dependencies) and all roles are gzipped and uploaded to that node, to the `/var/chef-solo/` directory. A node.json file gets created on the fly and uploaded, and Chef Solo gets executed at the remote node, using node.json as the node configuration and the pre-installed solo.rb for Chef Solo configuration.
+Whenever you apply a recipe to a node, all needed cookbooks (including dependencies), all roles and all databags are gzipped and uploaded to that node, to the `/var/chef-solo/` directory. A node.json file gets created on the fly and uploaded, and Chef Solo gets executed at the remote node, using node.json as the node configuration and the pre-installed solo.rb for Chef Solo configuration.
-The result is that you can play as often with your recipes and nodes as you want, without having to worry about a central Chef repository, Chef server nor anything else. You can make small changes to your cookbooks and test them again and again without having to commit the changes. You commit to your repo only when you want. LittleChef brings back sanity to cookbook development.
+The result is that you can play as often with your recipes and nodes as you want, without having to worry about a central Chef repository, Chef server nor anything else. You can make small changes to your cookbooks and test them again and again without having to commit the changes. You commit to your repo only when you want. LittleChef brings sanity to cookbook development.
## Installation
@@ -54,7 +55,7 @@ Careful what you do with your nodes!:
### Local Setup
-`cook new_kitchen` will create inside the current directory a few files and directories for LittleChef to be able to cook: `auth.cfg`, `roles/`, `nodes/`, `cookbooks/` and `site-cookbooks/`. You can create and have as many kitchens as you like on your computer.
+`cook new_kitchen` will create inside the current directory a few files and directories for LittleChef to be able to cook: `auth.cfg`, `roles/`, `data_bags/`, `nodes/`, `cookbooks/` and `site-cookbooks/`. You can create and have as many kitchens as you like on your computer.
### Authentication
@@ -130,13 +131,13 @@ You can import littlechef.py into your own Python project. The following
script is equivalent to using the `cook` orders:
```python
-from littlechef import littlechef
-littlechef.env.user = 'MyUsername'
-littlechef.env.password = 'MyPassword'
-littlechef.env.host_string = 'MyHostnameOrIP'
-littlechef.deploy_chef(gems='yes', ask='no')
-littlechef.recipe('MYRECIPE')#Applies <MYRECIPE> to <MyHostnameOrIP>
-littlechef.configure()#Applies again the saved nodes/MyHostnameOrIP.json configuration
+from littlechef import littlechef as lc
+lc.env.user = 'MyUsername'
+lc.env.password = 'MyPassword'
+lc.env.host_string = 'MyHostnameOrIP'
+lc.deploy_chef(gems='yes', ask='no')
+lc.recipe('MYRECIPE')#Applies <MYRECIPE> to <MyHostnameOrIP>
+lc.configure()#Applies the saved nodes/MyHostnameOrIP.json configuration
```
### Other tutorial material
@@ -158,7 +159,8 @@ Happy cooking!
[Nodes]: http://wiki.opscode.com/display/chef/Nodes
[Cookbooks]: http://wiki.opscode.com/display/chef/Cookbooks
[Roles]: http://wiki.opscode.com/display/chef/Roles
+ [Data Bags]: http://wiki.opscode.com/display/chef/Data+Bags
[Opscode repository]: http://wiki.opscode.com/display/chef/Installation#Installation-InstallingChefClientandChefSolo:
[Automated Deployments with LittleChef]: http://sysadvent.blogspot.com/2010/12/day-9-automated-deployments-with.html
[discussion group]: http://groups.google.com/group/littlechef
- [https://github.com/tobami/littlechef/issues]: https://github.com/tobami/littlechef/issues
+ [https://github.com/tobami/littlechef/issues]: https://github.com/tobami/littlechef/issues
View
37 littlechef/chef.py
@@ -26,10 +26,7 @@
import littlechef
import lib
-
-
-# Upload sensitive files with secure permissions
-_file_mode = 400
+import solo
def _save_config(node):
@@ -41,7 +38,7 @@ def _save_config(node):
files_to_create = ['tmp_node.json']
if not os.path.exists(filepath):
# Only save to nodes/ if there is not already a file
- print "Saving configuration to {0}".format(filepath)
+ print "Saving node configuration to {0}...".format(filepath)
files_to_create.append(filepath)
for node_file in files_to_create:
with open(node_file, 'w') as f:
@@ -61,9 +58,7 @@ def sync_node(node):
def _build_node(node):
- """Performs the Synchronize step of a Chef run:
- Uploads needed cookbooks and all roles to a node
- """
+ """Builds a list with all needed cookbooks and their dependencies"""
cookbooks = []
# Fetch cookbooks needed for recipes
for recipe in lib.get_recipes_in_node(node):
@@ -113,6 +108,8 @@ def _build_node(node):
def _synchronize_node(cookbooks):
+ """Performs the Synchronize step of a Chef run:
+ Uploads needed cookbooks, all roles and all databags to a node"""
# Clean up node
for path in ['roles'] + littlechef.cookbook_paths:
with hide('stdout'):
@@ -128,20 +125,38 @@ def _synchronize_node(cookbooks):
print " ({0})".format(", ".join(c for c in cookbooks))
to_upload = [p for p in cookbooks_by_path.keys()]
to_upload.append('roles')
+ to_upload.append('data_bags')
_upload_and_unpack(to_upload)
+ _add_data_bag_patch()
+
+
+def _add_data_bag_patch():
+ """Adds data_bag_lib cookbook, which provides a library to read data bags"""
+ # Create extra cookbook dir
+ lib_path = os.path.join(
+ littlechef.node_work_path, littlechef.cookbook_paths[0],
+ 'data_bag_lib', 'libraries')
+ sudo('mkdir -p {0}'.format(lib_path))
+ # Path to local patch
+ basedir = os.path.abspath(os.path.dirname(__file__).replace('\\','/'))
+ # Create remote data bags patch
+ put(os.path.join(basedir, 'data_bags_patch.rb'),
+ os.path.join(lib_path, 'data_bags.rb'), use_sudo=True)
def _configure_node(configfile):
"""Exectutes chef-solo to apply roles and recipes to a node"""
with hide('running'):
print "Uploading node.json..."
remote_file = '/root/{0}'.format(configfile.split("/")[-1])
- put(configfile, remote_file, use_sudo=True, mode=_file_mode)
+ # Ensure secure permissions
+ put(configfile, remote_file, use_sudo=True, mode=400)
sudo('chown root:root {0}'.format(remote_file)),
sudo('mv {0} /etc/chef/node.json'.format(remote_file)),
# Remove local temporary node file
os.remove(configfile)
-
+ # Always configure Chef Solo
+ solo.configure()
print "\n== Cooking ==\n"
with settings(hide('warnings'), warn_only=True):
output = sudo(
@@ -187,7 +202,7 @@ def _upload_and_unpack(source):
'cd tmp && COPYFILE_DISABLE=true tar czf ../{0} --exclude=".svn" .'.format(
local_archive))
# Upload archive to remote
- put(local_archive, remote_archive, use_sudo=True, mode=_file_mode)
+ put(local_archive, remote_archive, use_sudo=True, mode=400)
# Remove local copy of archive and directory
local('rm {0}'.format(local_archive))
local('chmod -R u+w tmp')
View
45 littlechef/data_bags_patch.rb
@@ -0,0 +1,45 @@
+# save this in the library folder of a cookbook
+# (e.g. ./coookbooks/vagrant/library/chef_solo_patch.rb)
+# see also https://gist.github.com/867958 for vagrant patch
+# Taken from Vagrant's patch: https://gist.github.com/867960
+# based on http://lists.opscode.com/sympa/arc/chef/2011-02/msg00000.html
+if Chef::Config[:solo]
+ class Chef
+ module Mixin
+ module Language
+ def data_bag(bag)
+ @solo_data_bags = {} if @solo_data_bags.nil?
+ unless @solo_data_bags[bag]
+ @solo_data_bags[bag] = {}
+ data_bag_path = Chef::Config[:data_bag_path]
+ Dir.glob(File.join(data_bag_path, bag, "*.json")).each do |f|
+ item = JSON.parse(IO.read(f))
+ @solo_data_bags[bag][item['id']] = item
+ end
+ end
+ @solo_data_bags[bag].keys
+ end
+
+ def data_bag_item(bag, item)
+ data_bag(bag) unless ( !@solo_data_bags.nil? && @solo_data_bags[bag])
+ @solo_data_bags[bag][item]
+ end
+
+ end
+ end
+ end
+
+ class Chef
+ class Recipe
+ def search(bag_name, query=nil)
+ Chef::Log.warn("Simplistic search patch, ignoring query of %s" % [query]) unless query.nil?
+ data_bag(bag_name.to_s).each do |bag_item_id|
+ bag_item = data_bag_item(bag_name.to_s, bag_item_id)
+ yield bag_item
+ end
+ end
+
+ end
+ end
+
+end
View
9 littlechef/littlechef.py
@@ -53,6 +53,7 @@ def new_kitchen():
def _mkdir(d):
if not os.path.exists(d):
os.mkdir(d)
+ # Add an empty README so that it can be added to version control
readme_path = os.path.join(d, 'README')
if not os.path.exists(readme_path):
with open(readme_path, "w") as readme:
@@ -61,8 +62,10 @@ def _mkdir(d):
_mkdir("nodes")
_mkdir("roles")
+ _mkdir("data_bags")
for cookbook_path in cookbook_paths:
_mkdir(cookbook_path)
+ # Add skeleton auth.cfg
if not os.path.exists("auth.cfg"):
with open("auth.cfg", "w") as authfh:
print >> authfh, "[userinfo]"
@@ -108,7 +111,8 @@ def deploy_chef(gems="no", ask="yes", version="0.9"):
def recipe(recipe):
- """Apply the given recipe to a node, ignoring any existing configuration
+ """Apply the given recipe to a node
+ Sets the run_list to the given recipe
If no nodes/hostname.json file exists, it creates one
"""
# Check that a node has been selected
@@ -124,7 +128,8 @@ def recipe(recipe):
def role(role):
- """Apply the given role to a node, ignoring any existing configuration
+ """Apply the given role to a node
+ Sets the run_list to the given role
If no nodes/hostname.json file exists, it creates one
"""
# Check that a node has been selected
View
18 littlechef/solo.py
@@ -42,19 +42,23 @@ def install(distro_type, distro, gems, version):
def configure():
"""Deploy chef-solo specific files"""
with credentials():
- sudo('mkdir -p {0}'.format(littlechef.node_work_path))
sudo('mkdir -p {0}/cache'.format(littlechef.node_work_path))
sudo('umask 0377; touch solo.rb')
- append('solo.rb', 'file_cache_path "{0}/cache"'.format(
- littlechef.node_work_path), use_sudo=True)
+ text = ""
+ text += 'file_cache_path "{0}/cache"'.format(littlechef.node_work_path)
+ text += "\n"
reversed_cookbook_paths = littlechef.cookbook_paths[:]
reversed_cookbook_paths.reverse()
cookbook_paths_line = 'cookbook_path [{0}]'.format(', '.join(
- ['''"{0}/{1}"'''.format(littlechef.node_work_path, x) \
+ ['"{0}/{1}"'.format(littlechef.node_work_path, x) \
for x in reversed_cookbook_paths]))
- append('solo.rb', cookbook_paths_line, use_sudo=True)
- append('solo.rb', 'role_path "{0}/roles"'.format(
- littlechef.node_work_path), use_sudo=True)
+ text += cookbook_paths_line + "\n"
+ text += cookbook_paths_line + "\n"
+ text += 'role_path "{0}/roles"'.format(
+ littlechef.node_work_path) + "\n"
+ text += 'data_bag_path "{0}/data_bags"'.format(
+ littlechef.node_work_path) + "\n"
+ append('solo.rb', text, use_sudo=True)
sudo('mkdir -p /etc/chef')
sudo('mv solo.rb /etc/chef/')
View
47 littlechef/tests.py
@@ -1,39 +1,52 @@
import unittest
import os
import shutil
+import json
from fabric.api import *
-import littlechef
import chef
+import lib
class BaseTest(unittest.TestCase):
def setUp(self):
"""Simulate we are inside a kitchen"""
- if os.path.exists('nodes'):
- shutil.rmtree('nodes')
- os.mkdir('nodes')
+ for d in ['nodes', 'roles', 'cookbooks']:
+ shutil.copytree('../tests/{0}'.format(d), d)
def tearDown(self):
- shutil.rmtree('nodes')
- os.remove('tmp_node.json')
+ for d in ['nodes', 'roles', 'cookbooks']:
+ shutil.rmtree(d)
+ if os.path.exists('tmp_node.json'):
+ os.remove('tmp_node.json')
class TestChef(BaseTest):
def test_save_config(self):
- """Should create tmp_node.json and a nodes/node.json config file"""
+ """Should create tmp_node.json and a nodes/testnode2.json config file"""
+ env.host_string = 'testnode2'
+ run_list = ["role[testrole]"]
+ chef._save_config({"run_list": run_list})
+ self.assertTrue(os.path.exists(os.path.join('nodes/', 'testnode2.json')))
+ with open('nodes/' + 'testnode2.json', 'r') as f:
+ data = json.loads(f.read())
+ self.assertEquals(data['run_list'], run_list)
+ # It should't overwrite existing config files
+ env.host_string = 'testnode'# This node exists
+ run_list = ["role[testrole]"]
+ chef._save_config({"run_list": run_list})
+ with open('nodes/' + 'testnode.json', 'r') as f:
+ data = json.loads(f.read())
+ # It should *NOT* have "testrole" assigned
+ self.assertEquals(data['run_list'], ["recipe[subversion]"])
+
+ def test_build_node(self):
+ """Should build cookbooks dependencies"""
env.host_string = 'testnode'
- chef._save_config({"run_list": ["role[testrole]"]})
- self.assertTrue(os.path.exists(os.path.join('nodes/', 'testnode.json')))
-
- #def test_build_node(self):
- #"""Should build cookbooks dependencies"""
- #env.host_string = 'testnode'
- #chef._save_config({"run_list": ["role[testrole]"]})
- #nodedata = {"run_list": ["role[testrole]"]}
- #chef._build_node(nodedata,
- #littlechef.cookbook_paths, littlechef.node_work_path)
+ cookbooks = chef._build_node(lib.get_node(env.host_string))
+ self.assertEquals(cookbooks, ['subversion'])
+ #TODO: add
if __name__ == "__main__":
View
2 littlechef/version.py
@@ -1,2 +1,2 @@
-VERSION = (0, 5, 5)
+VERSION = (0, 6, 0)
version = ".".join([str(x) for x in VERSION])
View
7 setup.py
@@ -12,6 +12,7 @@
keywords=["chef", "devops"],
install_requires=['fabric>=1.0.1', 'simplejson'],
packages=['littlechef'],
+ package_data={'littlechef': ['data_bags_patch.rb']},
scripts=['cook'],
classifiers=[
"Programming Language :: Python",
@@ -31,13 +32,13 @@
cookbooks and its dependencies are gzipped and uploaded to that node. A
node.json file gets created on the fly and uploaded, and Chef Solo gets
executed at the remote node, using node.json as the node configuration and the
-pre-installed solo.rb for Chef Solo configuration. Cookbooks and roles are
-configured to be found at (/var/chef-solo/).
+pre-installed solo.rb for Chef Solo configuration. Cookbooks, data bags and roles
+are configured to be found at (/var/chef-solo/).
The result is that you can play as often with your recipes and nodes as you
want, without having to worry about a central Chef repository, Chef server nor
anything else. You can make small changes to your cookbooks and test them again
-and again without having to commit the changes. LittleChef brings back sanity
+and again without having to commit the changes. LittleChef brings sanity
to cookbook development.
.. _Chef: http://wiki.opscode.com/display/chef/Home
View
10 tests.py
@@ -1,10 +0,0 @@
-import unittest
-
-class ImportTest(unittest.TestCase):
- def test_import(self):
- """Should import the littlechef module"""
- import littlechef
-
-
-if __name__ == "__main__":
- unittest.main()
View
10 tests/tests.py
@@ -29,7 +29,7 @@ def test_version(self):
"""Should output the correct Little Chef version"""
resp, error = self.execute(['../cook', '-v'])
self.assertEquals(error, "")
- self.assertTrue('LittleChef 0.5.' in resp)
+ self.assertTrue('LittleChef 0.6.' in resp)
def test_list_commands(self):
"""Should output a list of available commands"""
@@ -69,8 +69,12 @@ def test_no_metadata(self):
cookbooks_path = os.path.dirname(os.path.abspath(__file__))
bad_cookbook = os.path.join(cookbooks_path, 'cookbooks', 'bad_cookbook')
os.mkdir(bad_cookbook)
- resp, error = self.execute(['../cook', 'list_recipes'])
- os.rmdir(bad_cookbook)
+ try:
+ resp, error = self.execute(['../cook', 'list_recipes'])
+ except OSError:
+ self.fail("Couldn't execute '../cook'")
+ finally:
+ os.rmdir(bad_cookbook)
expected = 'Fatal error: Cookbook "bad_cookbook" has no metadata.json'
self.assertTrue(expected in error)

0 comments on commit d8f14f8

Please sign in to comment.