This is a sample release repository for BOSH that deploys a three tier LAMP application: a wordpress blog which consists of a number of apache servers running php & wordpress, fronted by nginx, and using one mysql database for storage.
This document includes a tutorial on how to create a BOSH release. If you want to move directly to deploying the sample release skip to the deployment instructions.
A BOSH release consists of jobs, packages, source code, and associated release metadata. Large binary files (aka "blobs") required for packaging can be stored externally in a blobstore and referenced within the release, obviating the need to store them inside the release repository. Additionally, in order to utilize a BOSH release you must create a deployment manifest.
For a more detailed explanation of BOSH terminology please refer to the BOSH Reference.
The first step is to initialize the release:
$ bosh init release bosh-sample-release
which creates the release scaffolding:
$ find bosh-sample-release
bosh-sample-release
bosh-sample-release/blobs
bosh-sample-release/config
bosh-sample-release/config/blobs.yml
bosh-sample-release/jobs
bosh-sample-release/packages
bosh-sample-release/src
For now we are going to concentrate on the contents of the following directories:
- src: this is the source code for the packages
- packages: these are the instructions on how to compile the source into binaries
- jobs: these are the scripts and configuration files required to run the packages
In order to create a BOSH release you will need to map your existing cloud service into "jobs", the fundamental building block of a BOSH deployment. Essentially this is just creating logical groupings of processes that are organized around providing discrete functionality. Reviewing your architecture diagram is a good place to start.
In the case of our sample release, we are implementing an architecture with four easily recognizable roles: a web application (1) sitting behind a proxy (2), leveraging a database server (3) for dynamic content and a shared filesystem (4) for static content.
For each job you will need to identify:
- processes that need to run
- packages that are required for running these processes
- package dependencies (e.g. libraries)
The first job of our release is the wordpress web application, which will be served up by the Apache webserver. For now we'll work under the assumption that apache will be the only process we need to control for this job. Now we need to determine what BOSH packages need to be created for apache to serve wordpress.
It is important to note that when we refer to a package we are referring to BOSH packages. Do not confuse BOSH packages with "packages" installed by your operating system's package manager. The OS package manager is generally only used to manage the components of the BOSH stemcell.
Obviously, our first package is the apache webserver. The wordpress app is another package. Wordpress is written in PHP, so we have a dependency on the apache PHP module. Our database backend is mysql, so our PHP will have a dependency on mysql libraries being available. Our architecture also uses a shared filesystem, but we don't need to package the NFS client because it is available as part of the stemcell.
For the first job (wordpress) we have identified one process (apache) and four packages. Fortunately, the remaining jobs (nginx, mysql, nfs server) are fairly simple. They each essentially run a single process and do not have dependency chains, leaving us with the following:
jobs
- wordpress (web application)
- nginx (proxy)
- mysql (db)
- nfs server (shared filesystem)
packages
- apache (wordpress)
- wordpress (wordpress)
- php (wordpress)
- mysql (wordpress, mysql)
- nginx (nginx)
- nfs server (nfs server)
Once you have enumerated your jobs and pacakges, the next step is to start building packages.
First, run the BOSH command to create the package skeletons
e.g. apache2
$ cd bosh-sample-release
$ bosh generate package apache2
create packages/apache2
create packages/apache2/packaging
create packages/apache2/pre_packaging
create packages/apache2/spec
Generated skeleton for `apache2' package in `packages/apache2'
Then all other packages
$ echo common debian_nfs_server mysql nginx php5 wordpress | xargs -n1 bosh generate package
The spec file is a yaml document that lists the names of the required source files and other packages which are compilation dependencies.
The spec file for our apache package is quite simple:
packages/apache2/spec
---
name: apache2
files:
- apache2/httpd-2.2.25.tar.gz
The spec file for PHP is a bit more interesting, as the apache and mysql packages are compile time dependencies. Additionally, since the mysql source is separated into server/client components we've broken mysql up into two separate packages (mysql and mysqlclient).
packages/php5/spec
---
name: php5
dependencies:
- apache2
- mysqlclient
files:
- php5/php-5.3.10.tar.gz
You'll notice that the location of the source in the package spec is a relative path. This is because the file can either be a part of the repo (inside the src directory) or stored in the blobstore (synched to blobs directory). In the case of our sample release we're storing source in the repo:
$ mkdir src/apache2
$ cd src/apache2
$ wget 'http://apache.tradebit.com/pub/httpd/httpd-2.2.25.tar.gz'
Resolving apache.tradebit.com... 71.6.202.162
Connecting to apache.tradebit.com|71.6.202.162|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 7445734 (7.1M) [application/x-gzip]
Saving to: `httpd-2.2.25.tar.gz'
100%[======================================>] 7,445,734 798K/s in 9.7s
TODO: blob example
At compile time BOSH executes the packaging script. The source code will be located in the current working directory and can also be referenced by the environment variable BOSH_COMPILE_TARGET. Your script should install the compiled package underneath the BOSH_INSTALL_TARGET. If you are installing software that uses GNU autoconf your packaging script can often be as simple as untar-ing your source, running configure with the prefix set to BOSH_INSTALL_TARGET, followed by a "make install".
For example, our apache packaging script is a perfect example of a simple configure/install process:
packages/apache2/packaging
cat <<"EOF" >packages/apache2/packaging
echo "Extracting apache https..."
tar xzf apache2/httpd-2.2.25.tar.gz
echo "Building apache https..."
(
cd httpd-2.2.25
./configure \
--prefix=${BOSH_INSTALL_TARGET} \
--enable-so
make
make install
)
EOF
Due to the compile time dependencies PHP is more complicated:
packages/php5/packaging
cat <<"EOF" >packages/php5/packaging
echo "Extracting php5..."
tar xjf php5/php-5.3.27.tar.bz2
# ./configure needs the location of the apache and mysql packages
export APACHE2=/var/vcap/packages/apache2
echo "Building php5..."
cd php-5.3.27
./configure \
--prefix=${BOSH_INSTALL_TARGET} \
--with-apxs2=${APACHE2}/bin/apxs \
--with-config-file-path=/var/vcap/jobs/wordpress/config/php.ini \
--with-mysql=/var/vcap/packages/mysqlclient
EOF
stdout from make is redirected to /dev/null because the output is too large for the nats message bus
cat <<EOF >>packages/php5/packaging
make > /dev/null
make install > /dev/null
EOF
finally, make installs into the apache module directory, we have to manually copy it back into BOSH_INSTALL_TARGET otherwise it will be left behind on the compilation vm
cat <<"EOF" >>packages/php5/packaging
mkdir -p ${BOSH_INSTALL_TARGET}/modules
cp ${APACHE2}/modules/libphp5.so ${BOSH_INSTALL_TARGET}/modules
EOF
Now that all of the packages have been created we can start assembling jobs.
First, run the BOSH command to create the job skeleton:
$ cd bosh-sample-release/
$ bosh generate job wordpress
create jobs/wordpress
create jobs/wordpress/templates
create jobs/wordpress/spec
create jobs/wordpress/monit
Generated skeleton for `wordpress' job in `jobs/wordpress'
The spec file is a yaml document that defines
- required packages
- paths to the configuration templates
- job properties (plus descriptions and defaults) used by the templates
The nginx job has a fairly straightforward spec:
jobs/nginx/spec
cat <<"EOF" >jobs/nginx/spec
---
name: nginx
templates:
nginx_ctl: bin/nginx_ctl
nginx.conf.erb: config/nginx.conf
mime.types: config/mime.types
packages:
- nginx
properties:
nginx.workers:
description: Number of nginx worker processes
default: 1
wordpress.servername:
description: Name of the virtual server
wordpress.port:
description: TCP port upstream (backends) servers listen on
default: 8008
wordpress.servers:
description: Array of upstream (backends) servers
EOF
The first step toward creating a deployable job is defining the package dependencies in the spec file. As we determined earlier, the wordpress job requires the apache, wordpress, php, and mysql client packages:
jobs/wordpress/spec
cat <<"EOF" >jobs/wordpress/spec
---
name: wordpress
packages:
- mysqlclient
- apache2
- php5
- wordpress
EOF
BOSH uses monit to manage the lifecycle of job processes. At a bare minimum the monit config file inside the job directory needs to define a pidfile and how to start/stop the job. It's going to be a lot easier to build out the rest of our job after we've got a vm deployed with packages installed, so we're going to create a stub monit config:
jobs/wordpress/monit
cat <<"EOF" >jobs/wordpress/monit
check process wordpress
with pidfile /var/run/stub.pid
start program "/bin/cp /var/run/monit.pid /var/run/stub.pid" with timeout 60 seconds
stop program "/bin/rm -f /var/run/stub.pid" with timeout 60 seconds
group vcap
EOF
and update our spec:
jobs/wordpress/spec
cat <<"EOF" >jobs/wordpress/spec
---
name: wordpress
templates: {}
packages:
- mysqlclient
- apache2
- php5
- wordpress
EOF
This will allow us to define a 'wordpress' job in our deployment manifest and deploy a vm with our packages.
The templates directory contains the scripts and config files needed for running the job processes. BOSH evaluates these files as ruby ERB templates, allowing them to be generalized. The unique per-deployment information can then be abstracted into properties defined in the deployment manifest.
It generally easier to create templates on an existing BOSH instance that has the required packages already installed. Start with the configuration files needed by the job packages, and once the processes can be launched manually then move on to writing the control scripts. When converting existing files/scripts to work on a BOSH managed instance, the most common modifications are usually changing paths to the appropriate location on a BOSH managed system.
BOSH filesystem conventions
object | path |
---|---|
package contents | /var/vcap/package/{package name} |
job configuration files | /var/vcap/jobs/{job name}/config |
control scripts | /var/vcap/jobs/{job name}/bin |
pidfiles | /var/vcap/sys/run |
log storage | /var/vcap/sys/log/{process name} |
persistent data storage | /var/vcap/store |
For example, in the wordpress job the apache configuration needs to set the document root to point at our wordpress package:
DocumentRoot "/var/vcap/packages/wordpress"
and for the appropriate log file locations:
ErrorLog "/var/vcap/sys/log/apache2/error_log"
CustomLog "/var/vcap/sys/log/apache2/access_log" common
Once the configuration file is completed it can be copied into the templates directory and the spec file updated:
jobs/wordpress/spec
cat <<"EOF" >jobs/wordpress/spec
---
name: wordpress
templates:
httpd.conf.erb: config/httpd.conf
packages:
- mysqlclient
- apache2
- php5
- wordpress
EOF
Once all of the necessary configuration files are in place you must write a control script that monit can use to start/stop the process and replace the stub monit config.
TODO: detailed control script example
At this point you should have a job that can be deployed by BOSH. The final step is to determine what configuration information should be abstracted into job properties. Passwords, account names, shared secrets, hostnames, IP addresses and port numbers are all examples of typical job properties.
To reference a job property in a ERB template, use the "p" helper, e.g.:
jobs/wordpress/templates/wp-config.php.erb
cat <<"EOF" >jobs/wordpress/templates/wp-config.php.erb
define('AUTH_KEY', '<%= p("wordpress.auth_key") %>');
define('SECURE_AUTH_KEY', '<%= p("wordpress.secure_auth_key") %>');
define('LOGGED_IN_KEY', '<%= p("wordpress.logged_in_key") %>');
define('NONCE_KEY', '<%= p("wordpress.nonce_key") %>');
define('AUTH_SALT', '<%= p("wordpress.auth_salt") %>');
define('SECURE_AUTH_SALT', '<%= p("wordpress.secure_auth_salt") %>');
define('LOGGED_IN_SALT', '<%= p("wordpress.logged_in_salt") %>');
define('NONCE_SALT', '<%= p("wordpress.nonce_salt") %>');
EOF
All job properties should be defined in your job spec, along with a description and an optional default value.
jobs/wordpress/spec
cat <<"EOF" >>jobs/wordpress/spec
properties:
wordpress.auth_key:
description: Wordpress Authentication Unique Keys (AUTH_KEY)
wordpress.secure_auth_key:
description: Wordpress Authentication Unique Keys (SECURE_AUTH_KEY)
wordpress.logged_in_key:
description: Wordpress Authentication Unique Keys (LOGGED_IN_KEY)
wordpress.nonce_key:
description: Wordpress Authentication Unique Keys (NONCE_KEY)
wordpress.auth_salt:
description: Wordpress Authentication Unique Salts (AUTH_SALT)
wordpress.secure_auth_salt:
description: Wordpress Authentication Unique Salts (SECURE_AUTH_SALT)
wordpress.logged_in_salt:
description: Wordpress Authentication Unique Salts (LOGGED_IN_SALT)
wordpress.nonce_salt:
description: Wordpress Authentication Unique Salts (NONCE_SALT)
EOF
To deploy the sample application edit the example deployment manifest (located at the /examples directory) and adapt it with your data. Then use the following command sequence:
bosh upload release releases/wordpress-2.yml
bosh deployment <wordpress deployment manifest>
bosh deploy
- Run through this readme and commit updated code to this repo
- Fill in all missing jobs specs etc. so this README is a complete set of steps to recreate this release from scratch
- When committing update code, recreate git history to match each step in the README?
- Another example bosh-release-simple: Use Redis
- Rename this example bosh-release-intermediate: Rename this release?
- bosh-release-advanced: Use HyperDB & MySQL master/slave replication
- tmux screencast of this README?