requirements | batteries included | toys in the toybox
optional stuff | tests | demos | implementation remarks | quick-links | todo | credits
##Toybox
Toybox is a template that builds an awesome development environment for virtualbox using vagrant and puppet. Apart from actually using the toybox as a development playground, this should be usable as pattern boilerplate for other kinds of custom automation. Toybox includes demos and integration tests for the server that is created, and the tests can be run from either the guest or host OS. ##Build target/reference versions:Vagrant 1.6.5 and Virtualbox 4.3.18 on the host with an Ubuntu guest. Note that currently only some random subset of the toys will work with redhat/centos. At least across recent Ubuntu versions, the puppet code is probably (possibly? maybe? hopefully??) fairly generic.
A known working base-box is Ubuntu 14.04 "trusty" (for download command, see "Usage" section). Download vagrant here, download virtualbox here.
##Basic batteries are included: * misc: git, ack-grep, nmap, screen, and tree * basic dev: * ruby: ruby 1.9.3, ruby-dev, gem * python: python 2.7.6, python-pip, python-dev, python-virtualenv ##Other toys in the toybox * [mongodb](http://www.mongodb.org): a popular nosql database * default port @ 27017 * mongo version is 2.4.9 * started by default on system boot * [genghisapp](http://genghisapp.com): a data viz tool for mongo * version @ 2.3.11 * WUI port at 5556 * see Vagrantfile to check if port-forwarding is enabled * [rabbitmq](https://www.rabbitmq.com): a message queue * Erlang R16B03 will be installed * data port @ 5672 * rabbit version is 3.4.0 * users: admin/admin, guest/guest * WUI port at 15672 * see Vagrantfile to check if port-forwarding is enabled * [celery](http://celery.readthedocs.org): task queue framework (uses rabbit) * celery version is 3.1.6 * no workers are defined or started by default * [flower](http://flower.readthedocs.org/en/latest/): a data viz tool for celery * flower version is 0.7.3 * WUI port @ 5555 * see Vagrantfile to check if port-forwarding is enabled * [supervisor](http://supervisord.org): a configurable, lightweight daemon tool * version @ 3.0 * responsible for daemonizing flower * responsible for daemonizing genghisapp * WUI port @ 9001 * see Vagrantfile to check if port-forwarding is enabled * [nginx](http://nginx.org/en/docs/): a webserver with fairly sane configs * version @ 1.4.6 * WUI/data port @ 80 * default config is simple: just a rendered version of this markdown * see Vagrantfile to check if port-forwarding to host 8080 is enabled##Optional toys:
- neo4j: a graph database
- WUI/data port @ 7474
- gDB kernel version is 1.7.2
- this will also install Java. sorry
- see Vagrantfile to check if port-forwarding is enabled
- XWindows and XMonad 0.11
- basic naked installation (well ok, and emacs23)
- you must use
startx
; will not startup on boot - the sole window manager (XMonad) will be used by default with "startx".
- xmonad implicitly requires haskell
First we'll need to download a couple of things. Feel free to skip steps below if you have already cloned this repository, if you already know that you have the right base-box downloaded, or if the vagrant plugin is already installed.
$ git clone https://github.com/mattvonrocketstein/toybox.git
$ cd toybox
$ vagrant box add trusty64 https://cloud-images.ubuntu.com/vagrant/trusty/current/trusty-server-cloudimg-i386-vagrant-disk1.box
$ vagrant plugin install vagrant-vbguest
Now we'll bring the box up and start the provisioning. This will take a long time while it imports the basebox, bootstraps it, and then runs the provisioning. Notes: Subsequent calls like the one below will not by default rerun provisioning. If vbguest detects a mismatched guest-additions iso, it may take even longer while it corrects this but it is better to fix it ASAP.
$ vagrant up
If the command above happens to terminate with a red message "The SSH command responded with a non-zero exit status. Vagrant assumes that this means the command failed", it might be caused by some download via apt, gem, or pip which has timed out. If that's the case, you may want to retry provisioning by executing vagrant provision
.
If you don't see the "non-zero exit status" message, then it probably succeeded. But just to be sure that puppet hasn't missed starting or restarting a service that it's updated, you should run
$ vagrant reload
After this, your box should be working. You can connect to it now, or try running the tests.
$ vagrant ssh
####Provisioning XWindows
If you wish to use the optional xwindows setup, I strongly suggest installing the vbguest plugin vbguest plugin. (Note: Unfortunately even then getting full-screen resolution to work may still take some extra fiddling, the situation seems to change slightly with every minor-version release of guest-extensions/virtualbox.) If you want to change the window manager or other details of this aspect of provisioning, fork this repo and edit toybox/puppet/modules/site/manifests/xwindows.pp
. To enable provisioning for xwindows, run:
$ PROVISION_XWIN=true vagrant provision
Provisioning neo is similar to provisioning XWindows, but you will need to download their distribution tarball first. Note: before you start the download, make sure that you're still in the same directory as this README and the Vagrantfile.
$ wget http://dist.neo4j.org/neo4j-community-1.7.2-unix.tar.gz
$ PROVISION_NEO=true vagrant provision
By default, the Vagrantfile forwards lots of ports for the services puppet is expected to bring up. During development it can be useful to verify that those services are indeed alive. To bootstrap the testing-setup on the host:
$ virtualenv host_venv
$ source host_venv/bin/activate
$ pip install -r tests/requirements.txt
$ python tests/test_guest.py
During normal provisioning, guest_venv
is setup automatically. To run tests on the guest from the guest, run this command from the host:
$ vagrant ssh -c "/vagrant/guest_venv/bin/python /vagrant/tests/test_guest.py"
$ vagrant ssh # connect to guest
$ source /vagrant/guest_venv/bin/activate # run this from guest
To run the celery/rabbit demo follows the instructions below. You can confirm the operations by watching graphs change in real time on your local flower and rabbitmq servers.
# send 1000 and 500 tasks to add and subtract worker, respectively
$ python /vagrant/demos/demo_celery.py --add -n 1000
$ python /vagrant/demos/demo_celery.py --add -n 500
# start a worker to deal with tasks
$ python /vagrant/demos/demo_celery.py --worker
To run the MongoDB demo follow the instructions below. You can confirm the operations by checking your local genghisapp, specifically the user collection.
# create 50 fake users
$ python /vagrant/demos/demo_mongo.py --records 50
To run the Neo4j demo you must already have done some of the optional provisioning, and then you can follow the instructions below. If it's not present on the guest in the /vagrant directory, the example movies database will be downloaded and afterwards it will be loaded into your neo server. After loading a dataset, visit your local neo server. If you want to start over, you can flush the database by using the --wipedb
argument to the demo_neo.py
script. See the script code for other usage instructions.
# load default datset "cieasts_12k_movies_50k"
$ python /vagrant/demos/demo_neo.py
puppet/modules/site/manifests/my_code.pp
puppet/modules/site/files/nginx_conf/sites-enabled/default
puppet/modules/site/manifests/configuration.pp
puppet/modules/site/manifests/xwindows.pp
puppet/modules/core/manifests/toybox.pp
puppet/modules/core/manifests/basic_dev.pp