chapmanb/cloudbiolinux

Newer

Older

Raw Normal view History

100644 224 lines (143 sloc) 7.35 KB

Dec 19, 2011 Documentation edits
Dec 19, 2011 Documentation edits	1	# CloudBioLinux, VirtualBox and Vagrant
May 26, 2011 Adding documentation oustide README.md
	2
	3	This document gives some additional information on using Vagrant with BioLinux.
	4	[Vagrant][v1] is a convenient command line manager for VirtualBox. In conjunction
	5	with the BioLinux fabric environment, any VirtualBox VM can be bootstrapped.
	6
Nov 22, 2011 Reorganize virtualbox documentation; add details from Ntino on Virtua…
	7	Note the current version of vagrant needs at least VirtualBox version 4.1.x.
	8
	9	## VirtualBox with vagrant
	10
	11	Add a base image to vagrant, and boot it up; community Vagrant boxes are available from
Nov 22, 2011 Reorganize virtualbox documentation; add details from Ntino on Virtua…
	12	[http://vagrantbox.es][v3] and [BioLinux flavors][v4]:
Nov 22, 2011 Reorganize virtualbox documentation; add details from Ntino on Virtua…
	13
	14	vagrant box add box_name http://path_to_the_image.box
	15	mkdir tmp/biolinux
	16	cd tmp/biolinux
	17	vagrant init box_name
	18	vagrant up
	19
	20	Run the fabfile, building CloudBioLinux:
	21
	22	fab -H vagrant -f /path/to/cloudbiolinux/fabfile.py install_biolinux
	23
	24	Then build the box, renaming package.box to `cloudbiolinux_date` and
	25	move it to a public webserver, such as Amazon S3:
	26
	27	vagrant package
	28	mv package.box biolinux_20110122.box
	29	s3cmd put --acl-public --guess-mime-type biolinux_20110122.box
	30	s3://chapmanb/biolinux_20110122.box
	31
	32	[v3]: http://vagrantbox.es/
Nov 22, 2011 Reorganize virtualbox documentation; add details from Ntino on Virtua…
	33	[v4]: http://biobeat.org/bionode
Jul 14, 2011 Comments and notes
Jul 14, 2011 Comments and notes	34
May 26, 2011 Adding documentation oustide README.md
	35	# Rolling your own
	36
	37	## Start from a BioLinux box
	38
	39	See the main README file for firing up a pre-installed BioLinux box.
	40
	41	## Start from scratch
	42
Jul 3, 2011 doc texts
	43	Despite the extra work, starting from scratch may have advantages. For
	44	one you have more control of the base install. Say for a different
	45	version of Linux, a BSD kernel, or for install less software (do you
Jul 14, 2011 Starting on a testing frame work at Pre-BOSC hackaton
	46	really need X/KDE/Gnome?), so you do not end up with an 8 GB VM, or for more
	47	software and/or data pre-intstalled on a VM.
	48
Jul 15, 2014 Ripping out edition support - migrating that to Flavors
	49	The BioLinux setup is designed to be modular, to support multiple
Jul 14, 2011 Starting on a testing frame work at Pre-BOSC hackaton
	50	flavors (see the main README for an explanation of terms).
May 26, 2011 Minimal edition of BioLinux works
May 26, 2011 Minimal edition of BioLinux works	51
May 26, 2011 Adding documentation oustide README.md
	52	Start with a standard downloadable prepared Vagrant box. For example a Debian
	53	32-bits box prepared for Vagrant, or create one from scratch as is explained on
	54	the [Vagrant web site][v1].
	55
Jul 3, 2011 doc texts
Jul 3, 2011 doc texts	56	Next add the virtualbox to vagrant using a URL, or box file:
May 26, 2011 Adding documentation oustide README.md
May 26, 2011 Adding documentation oustide README.md	57
May 26, 2011 Vagrant
May 26, 2011 Vagrant	58	vagrant box add debian_squeeze_32 debian_squeeze_32.box
May 26, 2011 Adding documentation oustide README.md
May 26, 2011 Adding documentation oustide README.md	59
Oct 4, 2011 Flavors available for download
	60	(boxes are available form [http://vagrantbox.es][v3] and
	61	[http://biobeat.org/bionode][BioLinux flavors]) and create your own version
May 26, 2011 Adding documentation oustide README.md
May 26, 2011 Adding documentation oustide README.md	62
Jul 3, 2011 doc texts
	63	mkdir myflavor
	64	cd myflavor
May 26, 2011 Adding documentation oustide README.md
	65
	66	Creates a ./Vagrantfile describing the VM.
	67
May 26, 2011 Vagrant
May 26, 2011 Vagrant	68	vagrant init debian_squeeze_32
May 26, 2011 Adding documentation oustide README.md
May 26, 2011 Adding documentation oustide README.md	69
Jul 3, 2011 %s/install_bare/install_biolinux/g
	70	Have a look inside the Vagrantfile. The default should be fine now.
	71
May 26, 2011 Vagrant
May 26, 2011 Vagrant	72	Start the VM (which gets copied the first time, which may take a while):
May 26, 2011 Adding documentation oustide README.md
May 26, 2011 Adding documentation oustide README.md	73
May 26, 2011 Vagrant
May 26, 2011 Vagrant	74	vagrant up
May 26, 2011 Adding documentation oustide README.md
	75
	76	and login
	77
Jul 3, 2011 %s/install_bare/install_biolinux/g
Jul 3, 2011 %s/install_bare/install_biolinux/g	78	vagrant ssh # no password needed
May 26, 2011 Adding documentation oustide README.md
May 26, 2011 Adding documentation oustide README.md	79
Jul 3, 2011 %s/install_bare/install_biolinux/g
	80	make sure you have enough disk space (twice the box size) for the dir
	81	~/VirtualBox\ VMs and ~/.vagrant, as this is where VMs are copied from the
	82	original box file.
May 26, 2011 Adding documentation oustide README.md
May 26, 2011 Adding documentation oustide README.md	83
Jul 3, 2011 %s/install_bare/install_biolinux/g
	84	At this point a bare VM is running that will accept BioLinux installations. The
	85	next step is to pull the BioLinux tree on your local system, and to run fab using the
	86	vagrant host, using a minimal install target. E.g.
May 26, 2011 Adding documentation oustide README.md
May 26, 2011 Adding documentation oustide README.md	87
May 26, 2011 Vagrant
May 26, 2011 Vagrant	88	export source=/path/to/cloudbiolinux
May 26, 2011 Adding documentation oustide README.md
May 26, 2011 Adding documentation oustide README.md	89
May 26, 2011 Minimal edition of BioLinux works
May 26, 2011 Minimal edition of BioLinux works	90	and
May 26, 2011 Adding documentation oustide README.md
May 26, 2011 Adding documentation oustide README.md	91
Jul 15, 2014 Documentation
Jul 15, 2014 Documentation	92	fab -f $source/fabfile.py -H vagrant -c $source/contrib/flavor/minimal/fabricrc_debian.txt install_biolinux:packagelist=$source/contrib/flavor/minimal/main.yaml
May 26, 2011 Adding documentation oustide README.md
May 26, 2011 Adding documentation oustide README.md	93
May 26, 2011 Vagrant
May 26, 2011 Vagrant	94	which uses the information from the local ./Vagrantfile.
May 26, 2011 Adding documentation oustide README.md
May 26, 2011 Adding documentation oustide README.md	95
May 26, 2011 Vagrant
	96	The first time the minimal fabfile is run it updates the /etc/apt/sources (on
	97	Debian-based systems), and a number of basic packages, including sudo, python,
	98	chef. It may be the Linux kernel and support libraries get upgraded, if they
	99	are in the dependency tree. Starting from a minimalistic Debian Vagrant box, the
	100	BioLinux minimal install has an unpacked size under 1Gb. E.g.
May 26, 2011 Adding documentation oustide README.md
May 26, 2011 Adding documentation oustide README.md	101
May 26, 2011 Vagrant
	102	vagrant@vagrant-debian-squeeze:~$ df -h
	103	Filesystem Size Used Avail Use% Mounted on
	104	/dev/sda1 39G 804M 36G 3% /
	105	tmpfs 188M 0 188M 0% /lib/init/rw
	106	udev 184M 116K 184M 1% /dev
	107	tmpfs 188M 0 188M 0% /dev/shm
	108	v-root 51G 24G 27G 47% /vagrant
	109
	110	Despite the fact that running fabfile.py is destructive, i.e. it overwrites the
	111	current install, it is reasonably safe as it mostly uses the underlying package
	112	management system and dependency resolution. Rerunning a BioLinux fabfile can
Jul 15, 2014 Ripping out edition support - migrating that to Flavors
	113	be fast. Minimal runs the second time in under 20 seconds on a
Oct 4, 2011 Documentation on creating and packaging a Vagrant Virtualbox VM
	114	basic laptop, as we do with a 'Minimal' install:
	115
	116	./test/test_vagrant --continue
May 26, 2011 Vagrant
	117
	118	For completeness, after a minimal install you can still install a full BioLinux
	119	execute
	120
May 28, 2011 Adding a full post_install example to a Flavor + documentation
	121	fab -H vagrant -f $source/fabfile.py install_biolinux
May 26, 2011 Vagrant
May 26, 2011 Vagrant	122
Oct 4, 2011 Documentation on creating and packaging a Vagrant Virtualbox VM
	123	Once you have a working Virtual Box VM with vagrant, you can package it with
	124
	125	vagrant package
	126
	127	and make the resulting .box file available for others to use.
	128
May 26, 2011 Vagrant
May 26, 2011 Vagrant	129	Read the README for further information.
May 26, 2011 Minimal edition of BioLinux works
	130
	131	[v1]: http://vagrantup.com/docs/base_boxes.html
Oct 7, 2011 Vagrant error message
	132
	133	## Trouble shooting
	134
	135	### Guest additions
	136
	137	You may see an error
	138
	139	[default] The guest additions on this VM do not match the install version of
	140	VirtualBox! This may cause things such as forwarded ports, shared
	141	folders, and more to not work properly. If any of those things fail on
	142	this machine, please update the guest additions and repackage the
	143	box.
	144
	145	Guest Additions Version: 4.0.4
	146	VirtualBox Version: 4.1.0
	147
	148	this error may actually be caused by the Vbox Linux kernel drivers not having
	149	been loaded! Fix
	150
	151	modprobe vboxdrv
	152
Nov 22, 2011 Reorganize virtualbox documentation; add details from Ntino on Virtua…
	153	# Converting Vagrant images to VirtualBox and Eucalyptus images
	154
	155	(protocol steps tested in Ubuntu Natty)
	156
	157	## software pre-requisite
	158
	159	sudo gem install vagrant
	160	sudo apt-get install cloud-utils
	161
	162	## Importing cloud biolinux VM to your system
	163
	164	vagrant box add base
	165	https://s3.amazonaws.com/cloudbiolinux/cbl_ubuntu_11_4_32_20110628.box
	166	vagrant init base
	167	vagrant up
	168
	169	## adding some missing components to the vagrant VMs
	170
	171	vagrant ssh
	172	sudo apt-get install gdm cloud-utils openssh
	173	sudo useradd -d /home/ubuntu -m ubuntusudo passwd ubuntu
	174	sudo shutdown -r now
	175
	176	in the graphical login after reboot get in with user:ubuntu / pass:ubuntu
	177	go to System--->Administration--->Login Window to enable autologin
	178
Dec 19, 2011 Documentation edits
Dec 19, 2011 Documentation edits	179	## VirtualBox Appliance
Nov 22, 2011 Reorganize virtualbox documentation; add details from Ntino on Virtua…
	180
Dec 20, 2011 Doc: Virtual appliance edit
	181	Virtual Appliances are pre-assemblied VM images configured for various purposes.
	182
	183	Open the Virtualbox GUI, you should see the VM added by vagrant - you can
Nov 22, 2011 Reorganize virtualbox documentation; add details from Ntino on Virtua…
	184	rename it to "Cloud BioLinux 32"
Dec 20, 2011 Doc: Virtual appliance edit
	185
	186	File->Export Appliance
	187
	188	and distribute the .ova.
	189
	190	Anyone in any OS running Virtualbox can import the .ova with File->Import
	191	Appliance.
Nov 22, 2011 Reorganize virtualbox documentation; add details from Ntino on Virtua…
	192
Dec 19, 2011 Documentation edits
Dec 19, 2011 Documentation edits	193	# Making a Eucalyptus image from VirtualBox
Nov 22, 2011 Reorganize virtualbox documentation; add details from Ntino on Virtua…
	194
Dec 19, 2011 Documentation edits
	195	Start with the Cloud BioLinux Virtualbox .vmdk (its location is in the VM
	196	properties from the Virtualbox GUI). Resize the vmdk, since the size may be
	197	40G, and the Eucalyptus image will have that size.
Nov 22, 2011 Reorganize virtualbox documentation; add details from Ntino on Virtua…
	198
Dec 19, 2011 Documentation edits
Dec 19, 2011 Documentation edits	199	Accordint to http://mtnbike.org/blog/?p=29 and the same here:
Nov 22, 2011 Reorganize virtualbox documentation; add details from Ntino on Virtua…
	200	http://www.my-guides.net/en/content/view/122/26/
	201
Dec 19, 2011 Documentation edits
Dec 19, 2011 Documentation edits	202	convert to raw .img
Nov 22, 2011 Reorganize virtualbox documentation; add details from Ntino on Virtua…
	203
	204	qemu-img convert -O raw CloudBioLinux-32bit-disk1.vmdk
	205	CloudBioLinux-32bit-disk1.img
	206
Dec 19, 2011 Documentation edits
Dec 19, 2011 Documentation edits	207	deploy to Eucalyptus via
Nov 22, 2011 Reorganize virtualbox documentation; add details from Ntino on Virtua…
	208
	209	uec-publish-img CloudBioLinux-32bit-disk1.img
Oct 7, 2011 Vagrant error message
Oct 7, 2011 Vagrant error message	210
Dec 19, 2011 Documentation edits
	211	# VirtualBox, KVM or XEN?
	212
	213	There are more ways than one to virtualize machines on Linux.
	214
	215	Despite the attractions of vagrant and Virtualbox, as displayed here, we note
	216	that Linux KVM may be a better choice for virtualization and testing of
	217	CloudBioLinux, as Linux distributions support KVM out of the box, and KVM has
	218	more Unix-like control. See also the information for using KVM in
	219	./doc/linux_kvm.md.
	220
	221	For production environments check out XEN virtualization (XEN runs Amazon EC2).
	222
	223