Skip to content

Commit

Permalink
work on vagrants -- setting up boxen on futhark
Browse files Browse the repository at this point in the history
  • Loading branch information
Philip (flip) Kromer committed Jan 20, 2012
1 parent 9bb2092 commit 7346d7f
Show file tree
Hide file tree
Showing 14 changed files with 325 additions and 152 deletions.
160 changes: 160 additions & 0 deletions notes/continuous_integration_testing.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@


https://github.com/acrmp/chefspec


pre-testing -- converge machine
https://github.com/acrmp/chefspec

http://wiki.opscode.com/display/chef/Knife#Knife-test

in-machine

* x ports on x interfaces open
* daemon is running
* file exists and has string

* log file is accumulating lines at rate X
* script x runs successfully

in-chef

* runlist is X
* chef attribute X should be Y

meta

* chef run was idempotent



__________________________________________________________________________

## Notes from around the web


* ...

> I'm thinking that the useful thing to test is NOT did chef install
> some package or setup a user, but rather after chef has run can I
> interact with the system as I would expect from an external
> perspective. For example:
>
> * Is the website accessible?
> * Are unused ports blocked?
> * When I send an email thorough the website does it end up in my inbox?
>
> Capybara (http://github.com/jnicklas/capybara) enforces this external
> perspective for webapp testing:
>
> "Access to session, request and response from the test is not
> possible. Maybe we’ll do response headers at some point in the future,
> but the others really shouldn’t be touched in an integration test
> anyway. "
>
> They only let you interact with screen elements that a user could
> interact with. It makes sense because the things that users interact
> with are what provides the business value
* Andrew Shafer < andrew@cloudscaling.com>

> Here's my thinking at this point... which could be wrong on every level.
> There is really no good way to TDD/BDD configuration management for several
> reasons:
> The recipes are already relatively declarative
> Mocking is useless because it may not reflect 'ground truth'
> The cycle times to really test convergence are relatively long
> Trying to test if a package is installed or not is testing the framework,
> not the recipe IMHO.
> I agree with the general sentiment that the functional service is the true
> test.
> I'm leaning towards 'testing' at that level, ideally with (a superset of?)
> what should be used for the production monitoring system.
> So the CI builds services, runs all the checks in test, green can go live
> and that's that.

* Jeremy Deininger < jeremy@rightscale.com>

> Thought I'd chime in with my experience testing system configuration code @ RightScale so far. What we've been building are integration style cucumber tests to run a cookbook through it's paces on all platforms and OSs that we support.
> First we use our API to spin up 'fresh' server clusters in EC2, one for every platform/OS (variation) that the cookbook will be supporting. The same could be done using other cloud APIs (anyone else doing this with VMware or etc?) Starting from scratch is important because of chef's idempotent nature.
> Then a cucumber test is run against every variation in parallel. The cucumber test runs a series of recipes on the cluster then uses what we call 'spot checks' to ensure the cluster is configured and functional. The spot checks are updated when we find a bug, to cover the bug. An example spot check would be, sshing to every server and checking the mysql.err file for bad strings.
> These high level integration tests are long running but have been very useful flushing out bugs.
> ...
> If you stop by the #rightscale channel on Freenode I'd be happy to embarrass myself by giving you a sneak peak at the features etc.. Would love to bounce ideas around and collaborate if you're interested. jeremydei on Freenode IRC
Ranjib Dey < ranjibd@th...s.com>

> So far, what we've done for testing is to use rspec for implementing tests. Here's an example:
>
> it "should respond on port 80" do
> lambda {
> TCPSocket.open(@server, 'http')
> }.should_not raise_error
> end
>
> Before running the tests, I have to manually bootstrap a node using knife. If my instance is the only one in its environment, the spec can find it using knife's search feature. The bootstrap takes a few minutes, and the 20 or so tests take about half a minute to run.
>
> While I'm iteratively developing a recipe, my work cycle is to edit source, upload a cookbook, and rerun chef-client (usually by rerunning knife boostrap, because the execution environment is different from invoking chef-client directly). This feels a bit slower than the cycle I'm used to when coding in Ruby because of the upload and bootstrap steps.
>
> I like rspec over other testing tools because of how it generates handy reports, such as this one, which displays an English list of covered test cases:
>
> $ rspec spec/ -f doc
>
> Foo role
> should respond on port 80
> should run memcached
> should accept memcached connections
> should have mysql account
> should allow passwordless sudo to user foo as user bar
> should allow passwordless sudo to root as a member of sysadmin
> should allow key login as user bar
> should mount homedirs on ext4, not NFS
> should rotate production.log
> should have baz as default vhost
> ...
>
> That sample report also gives a feel for sort of things we check. So far, nearly all of our checks are non-intrusive enough to run on a production system. (The exception is testing of local email delivery configurations.)
>
> Areas I'd love to see improvement:
>
> * Shortening the edit-upload-bootstrap-test cycle
> * Automating the bootstrap in the context of testing
> * Adding rspec primitives for Chef-related testing, which might
> include support for multiple platforms
>
> As an example of rspec primitives, instead of:
>
> it "should respond on port 80" do
> lambda {
> TCPSocket.open(@server, 'http')
> }.should_not raise_error
> end
>
> I'd like to write:
>
> it { should respond_on_port(80) }
>
> Rspec supports the the syntactic sugar; it's just a matter of adding some "matcher" plugins.
>
> How do other chef users verify that recipes work as expected?
>
> I'm not sure how applicable my approach is to opscode/cookbooks because it relies on having a specific server configuration and can only test a cookbook in the context of that single server. If we automated the boostrap step so it could be embedded into the rspec setup blocks, it would be possible to test a cookbook in several sample contexts, but the time required to setup each server instance might be prohibitive.
>

Andrew Crump < acrump@gmail.com>

> Integration tests that exercise the service you are building definitely give you the most bang for buck.
>
> We found the feedback cycle slow as well so I wrote chefspec which builds on RSpec to support unit testing cookbooks:
>
> https://github.com/acrmp/chefspec
>
> This basically fakes a convergence and allows you to make assertions about the created resources. At first glance Chef's declarative nature makes this less useful, but once you start introducing conditional execution I've found this to be a time saver.
>
> If you're looking to do CI (which you should be) converging twice goes some way to verifying that your recipes are idempotent.
>
> knife cookbook test is a useful first gate for CI:
>
> http://wiki.opscode.com/display/chef/Knife#Knife-test
1 change: 1 addition & 0 deletions vagrants/.gitignore
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
*.box
157 changes: 157 additions & 0 deletions vagrants/README-install.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@

### Install cluster_chef-homebase

Clone this repo, producing the directory we'll call `homebase` from now on. In fact, you may wish to rename it:

git clone https://github.com/infochimps-labs/cluster_chef-homebase
mv cluster_chef-homebase homebase
cd homebase
git submodule update --init

Now follow the instructions

* from [the main directory README.md](README.md), for overall setup
* from [the knife/ folder README.md](knife/README.md), for chef config file setup

you're ready to go when you can `knife cluster list` and get formatted output:

$ knife cluster list
+--------------------+---------------------------------------------------+
| cluster | path |
+--------------------+---------------------------------------------------+
| burninator | /cloud/clusters/burninator.rb |
| hadoop_demo | /cloud/clusters/hadoop_demo.rb |
... ...
| sandbox | /cloud/clusters/sandbox.rb |
+--------------------+---------------------------------------------------+


## Installing the Vagrant box for the first time:

### Install Virtualbox

Download and install Virtualbox 4.x -- visit http://download.virtualbox.org/virtualbox/

### Install the gems

Run bundle install from your homebase directory

$ cd cluster_chef_homebase
$ bundle install

You should now be able to list all templates:

$ vagrant basebox templates
The following templates are available:
# ....
vagrant basebox define '<boxname>' 'ubuntu-10.10-server-amd64'-netboot'
# ...
vagrant basebox define '<boxname>' 'ubuntu-11.10-server-amd64-ruby192'

### Build the new box

(_If you alread have the box in the vagrants/boxes folder, skip to 'Add the box'._)

Now move into the vagrants/ subdirectory and run

$ cd cluster_chef_homebase/vagrants
$ vagrant basebox build 'cluster-chef-natty'

If you don't have the iso file it will download it for you. The ISO file is huge, and will probably take about 30 minutes to pull in.

The `basebox build` command will

* create a machine + disk according to the definition.rb
* Note: :os_type_id = The internal Name Virtualbox uses for that Distribution
* Mount the ISO File :iso_file
* Boot up the machine and wait for `:boot_time`
* Send the keystrokes in `:boot_cmd`_sequence
* Startup a webserver on `:kickstart_port` to wait for a request for the `:kickstart_file` (don't navigate to the file in your browser or the server will stop and the installer will not be able to find your preseed)
* Wait for ssh login to work with :ssh_user , :ssh_password
* Sudo execute the :postinstall_files

Next, export the vm to a .box file (producing `cluster-chef-natty.box`)

$ vagrant basebox export cluster-chef-natty
$ mv cluster-chef-natty.box boxes/cluster-chef-natty.box

### Add the box as one of your boxes

Import the box into vagrant:

$ vagrant box add 'cluster-chef-natty' 'boxes/cluster-chef-natty.box'

__________________________________________________________________________

## Launch a VM with vagrant

To use it:

$ cd vagrants/cocina-chef_server
$ vagrant up
$ vagrant ssh

FIXME: this will break; it's trying to look in cloud/cookbooks/cookbooks
for cookbooks. ssh in, edit the /etc/chef/solo.rb file, and run chef-client by hand

__________________________________________________________________________

## Once your chef server is up

* visit the webui (at http://33.33.33.20:4040 by default).

* log in with username 'admin' and password p@ssw0rd1 -- it will immediately prompt you to change it.

* [Create an admin=true client](http://33.33.33.20:4040/clients/new) for
yourself. If you have an opscode platform user account, make life easy on
yourself and use the same name. Check the box 'yes' for admin

* Copy-paste the private key into `{homebase}/knife/cocina/{yourname}.pem`
- Do so immediately -- it's your only chance to get the key!
- The first and last lines of the file should be the `-----BEGIN...` and
`-----END..` blocks.
- do a `chmod 600 {homebase}/knife/cocina/{yourname}.pem`

* [Create the 'vm_dev' environment](http://33.33.33.20:4040/environments/new)
- you don't need to set anything, just create it.

* go to cluster_chef_homebase/knife and make a copy of the credentials directory for the cocina world

cd cluster_chef_homebase/knife
cp -rp example cocina
cd cocina
git init ; git add .
git commit -m "New credentials univers for local VM chef server" .

subdirectories of `cluster_chef_homebase/knife` are .gitignored; don't check this directory into git.

* upload your cookbooks!

cd /cloud
bundle install
export CHEF_USER=yourchefusername CHEF_ORGANIZATION=cocina CHEF_HOMEBASE=/cloud
rake roles & # NEVA GONNA GIVE YOU UP...
knife cookbook upload --all

### Make the chef_server a client of itself

* edit the file `cocina/cloud.rb`, and set

chef_server_url "http://33.33.33.20:4000/"

* ssh to the chef_server vm,

vagrant ssh

copy the server's copy of the validator so the machine can also be a client,
and grab a copy for posterity

cd /etc/chef
sudo ln -s validation.pem cocina-validator.pem
sudo cp validation.pem /cloud/knife/cocina/cocina-validator.pem
sudo mv dna.json /cloud/knife/cocina/dna/cocina-chef_server-0.json
sudo ln -s /cloud/knife/cocina/dna/cocina-chef_server-0.json dna.json
sudo ln -s /cloud/knife/cocina/client_keys client_keys

# when those finish,
sudo chef-client
Loading

0 comments on commit 7346d7f

Please sign in to comment.