Table of Contents generated with DocToc
- Dependencies - Installing chef - Installing librarian - Installing and configuring Vagrant - Preparing a new bare repo in git that will be configured via librarian/chef
- Configuration - Configuring chef using librarian - Some final configurations for Vagrant - Obtaining the cookbooks using librarian.
- Invoking the VM - Bringing up the dev environment - Accessing iPython notebook from the host machine - Distributing your project. - Creating your own cookbook for chef solo.
- Some useful links.
This setup has been tested with:
- OSX 10.7.5
- vagrant v1.2.7
- virtualbox 4.2.16
- chef 11.6.0
- ruby 2.0
This is a sample repo for spinning up a dev vm. We use librarian, Vagrant and chef solo and we will end up with a virtual machine with a number of python packages installed, running an instance of iPython Notebook that can be accessed from the host browser via port forwarding.
We also include instructions on how to create your own custom chef cookbooks that can be added in to the setup.
$ git clone git://github.com/elifesciences/elife-template-env.git $ cd elife-template-env $ librarian-chef install $ vagrant up $ vagrant ssh $ ipython notebook --ip=192.168.33.10
Then in your host browser open http://192.168.33.10:8888.
From the opscode wiki
sudo true && curl -L https://www.opscode.com/chef/install.sh | sudo bash
Librarian is a ruby gem that manages chef recepies.
$ gem install librarian
Vagrant needs a base box defined. Once defined it should download the machine image that will get reused. A nice list of images can be found at http://www.vagrantbox.es/. We are currently recommending using http://files.vagrantup.com/precise64.box.
After installing vagrant configure it as follows
$ vagrant box add precise64 http://files.vagrantup.com/precise64.box
Preparing a new bare repo in git that will be configured via librarian/chef
From within your bare repo initialise Vagrant.
$ vagrant init
This will generate a local
Now set up the git repo to ignore files that will be created during the running of librarian and chef.
$ echo cookbooks >> .gitignore $ echo tmp >> .gitignore $ git add .gitignore $ git commit -m "ignoring some files"
Configuring chef using librarian
Cheffile using librarian.
$ librarian-chef init
You now co-customise the
Cheffile to refer to the same cookbooks that you
require for your environment. The
Cheffile can actually contain more dependencies than
you need in your environment if you wish (for example if you have a generic set of tools and
you don't want to have custom
Cheffiles for each project).
Cheffile for this project currently looks like:
#!/usr/bin/env ruby #^syntax detection site 'http://community.opscode.com/api/v1' cookbook 'apt' cookbook 'git' cookbook 'python', '>= 1.2.2' cookbook 'rvm', :git => 'https://github.com/fnichol/chef-rvm' cookbook 'build-essential', :git => 'http://github.com/opscode-cookbooks/build-essential' cookbook 'elife-chef-dev-template', :git=> 'http://github.com/elifesciences/elife-chef-dev-template.git'
While the appropriate part of my
Vagrantfile looks like:
Vagrant::Config.run do |config| config.vm.provision :chef_solo do |chef| # This path will be expanded relative to the project directory chef.cookbooks_path = "cookbooks" chef.add_recipe "apt" chef.add_recipe "build-essential" chef.add_recipe "rvm::vagrant" chef.add_recipe "rvm::system" chef.add_recipe "git" chef.add_recipe "python" chef.add_recipe "elife-chef-dev-template" end
The important thing here is that for any package that you want to include in your VM, Vagrant needs to have that package specified, and the package also needs to be specified in the
Some final configurations for Vagrant
If you are running a web app, you want to be able to access the server in the VM from the host machine. This can be done via port forwarding. In addition set a static domain in the Vagrant file
config.vm.network :hostonly, "192.168.33.10"
Once you have configured these to your tastes you can commit them.
$ git add Vagrantfile $ git add Cheffile $ git commit -m "env setup"
Obtaining the cookbooks using librarian.
Cheffile describes which cookbooks we want to use.
librarian runs against the
Cheffile, gets the cookbooks, and sets them up locally with all path dependencies. Vagrant can then use the local cookbooks for configuring the VM.
$ librarian-chef install
Invoking the VM
Bringing up the dev environment
$ vagrant up
You can access the shared folder between the vagrant file system and the user system through
$ ls ~/vagrant
To stop the VM, in console:
$ vagrant suspend
To remove the VM, in console:
$ vagrant destroy
To reload the VM, in console:
$ vagrant reload
Accessing iPython notebook from the host machine
$ vagrant ssh $ ipython notebook --ip=192.168.33.10
You can now access your ipython notebook from the host machine by opening a browser at http://192.168.33.10:8888. It's important to pass the ip parameter to iPython, as it needs to be made aware of the endpoint that the VM will be using to communicat out of the box. This obviously needs to be set to be the same as the ip address that was set in the Vagrantfile. Enjoy!
Distributing your project.
You can commit the
Vagrantfile and the
Cheffile, push the changes to git hub and your dev environment is ready to distribute.
Creating your own cookbook for chef solo.
If you want to add a separate set of dependencies, or cookbook, then in a separate repo you should create a default Chef cookbook using
$ knife cookbook create cookbook-name
Create a github repo for this cookbook, and then point your
Cheffile at the repo, and update your
Vagrant file accordingly.
For this demo repo we have created an eLife specific cookbook that installs the following python dependencies:
Provisioning to AWS
Install the vagrant aws plugin
$ vagrant plugin install vagrant-aws
Configure the aws plugin with your info. It seems you need to create an AWS Ec2 instance, nominate the name of the ami, setup your local ssh credentials for accessing this ami, and then give vagrant these details, along with your aws keypairs. A good overview is provided here:
Bring up the machine in AWS
$ vagrant up --provider=aws
Vagrant won't shit all over your ports in use. The default Vagrant file in this repo asks vagrant to setup on port 8080, but if you have any other services running on this port Vagrant will not bring up the machine. Either kill your service, or change the ports in the Vagrantfile.
building a big machine
If you build a machine that has a lot of installs, vagrant can hang in the process. It looks like it doesn't work. Treat it like a computer science problem, kick it, turn it off and turn it on again and hope that it works.
- provide an example of cloning a repo into the newly created vm
- pass auth credentials either using a databag or some other method
- give an example of how to deploy the vm into AWS
Some useful links.