Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

Tutorial on scaling an environment with additional VMs and alternate VMs

  • Loading branch information...
commit dc21c5fd20c2db7793485c7b52ab5e5a733888f8 1 parent 763eb08
@drnic authored
View
5 changing-your-deployment.md
@@ -1,5 +0,0 @@
-# Changing your deployment
-
-The purpose of booting VMs, attaching disks and wiring up the network is to run jobs. Jobs on the internet (or intranet if its a private cloud).
-
-The sample release
View
144 examples/wordpress/deployment-manifest-scaling.yml
@@ -0,0 +1,144 @@
+---
+name: wordpress
+director_uuid: BOSH_DIRECTOR_UUID
+
+release:
+ name: wordpress
+ version: 1
+
+compilation:
+ workers: 2
+ network: default
+ cloud_properties:
+ disk: 8096
+ instance_type: m1.small
+
+# this section describes how updates are handled
+update:
+ canaries: 1
+ canary_watch_time: 30000
+ update_watch_time: 30000
+ max_in_flight: 4
+ max_errors: 1
+
+networks:
+ - name: default
+ type: dynamic
+ cloud_properties:
+ security_groups:
+ - default
+ - name: nginx
+ type: vip
+ cloud_properties:
+ security_groups:
+ - default
+ - name: wordpress
+ type: vip
+ cloud_properties:
+ security_groups:
+ - default
+ - name: mysql
+ type: vip
+ cloud_properties:
+ security_groups:
+ - default
+
+resource_pools:
+ - name: common
+ network: default
+ size: 4
+ stemcell:
+ name: bosh-stemcell
+ version: 0.5.1
+ cloud_properties:
+ disk: 8192
+ instance_type: m1.small
+ availability_zone:
+ key_name:
+ - name: mysql
+ network: default
+ size: 1
+ stemcell:
+ name: bosh-stemcell
+ version: 0.5.1
+ cloud_properties:
+ disk: 8192
+ instance_type: m1.xlarge
+ availability_zone:
+ key_name:
+
+jobs:
+ - name: nginx
+ template: nginx
+ instances: 1
+ resource_pool: common
+ networks:
+ - name: default
+ default: [dns, gateway]
+ cloud_properties:
+ instance_type: m1.small
+
+ - name: wordpress
+ template: wordpress
+ instances: 3
+ resource_pool: common
+ networks:
+ - name: default
+ default: [dns, gateway]
+ cloud_properties:
+ instance_type: m1.small
+
+ - name: mysql
+ template: mysql
+ instances: 1
+ resource_pool: mysql
+ persistent_disk: 16384
+ networks:
+ - name: default
+ default: [dns, gateway]
+ cloud_properties:
+ instance_type: m1.xlarge
+
+properties:
+
+ nginx:
+ workers: 1
+
+ wordpress:
+ admin: foo@bar.com
+ port: 8008
+ servers:
+ - WORDPRESS_ELASTICIP
+ servername: WORDPRESS_SERVERNAME
+ db:
+ name: wp
+ user: wordpress
+ pass: w0rdpr3ss
+ auth_key: random key
+ secure_auth_key: random key
+ logged_in_key: random key
+ nonce_key: random key
+ auth_salt: random key
+ secure_auth_salt: random key
+ logged_in_salt: random key
+ nonce_salt: random key
+
+ mysql:
+ address: MYSQL_ELASTICIP
+ port: 3306
+ password: rootpass
+
+cloud:
+ plugin: aws
+ properties:
+ aws:
+ access_key_id: ACCESS_KEY
+ secret_access_key: SECRET_ACCESS_KEY
+ default_key_name: fog_default
+ default_security_groups: ["default"]
+ ec2_endpoint: ec2.us-east-1.amazonaws.com
+ max_retries: 2
+ registry:
+ endpoint: http://admin:admin@BOSH_AWS_REGISTRY_DNS_NAME:25777
+ user: admin
+ password: admin
View
100 scaling-your-deployment.md
@@ -0,0 +1,100 @@
+# Scaling your deployment
+
+The purpose of booting VMs, attaching disks and wiring up the network is to run jobs. Jobs on the virtual machines. Each job can do a certain amount of work based on the compute resources you've given it. If you need more work done, then you can add more virtual machines or increase the size/attributes of the virtual machines you are using. With BOSH, this is very easy.
+
+NOTE: different jobs can run on the same virtual machine as each other. In the sample release tutorial we put the 3 jobs - nginx, wordpress and mysql on separate virtual machines.
+
+## Introduction to virtual machines on AWS
+
+On AWS, there are a fix set of virtual machines you can use. These are called "instance types". In the example deployment manifest, `wordpress-aws.yml`, we set `instance_type: m1.small` for all three jobs. After we deployed the release, if you looked in your AWS console you would see the 3 additional running VMs.
+
+There are several other [AWS instance types](http://aws.amazon.com/ec2/instance-types/ "Amazon EC2 Instance Types"). They are grouped together as families. For example, `m1.small`, `m1.medium`, `m1.large` and `m1.xlarge` are all in the "Normal" family. Each subsequent member of this family has 2x the CPU, RAM and I/O than its predecessor.
+
+For example, the `m1.small` has 1.7G of RAM and 1 virtual CPU (VCPU). The next member of the family, the `m1.medium`, has 3.75G of RAM and 2 VCPUS. You want 4 VCPUs and 7.5G of RAM? Then you want the `m1.large`. [AWS pricing](http://aws.amazon.com/ec2/pricing "Amazon EC2 Pricing") also doubles with each increase in instance type. As of writing, `m1.small` is 8c/hr, `m1.medium` is 16c/hr, etc.
+
+If your job requires a different ratio of CPU to RAM, then AWS has two other families of instance types - High CPU and High Memory.
+
+The High CPU family currently has two instance types - the `c1.medium` and `c1.xlarge`. The `c1.medium` is like the `m1.small` with 1.7G of RAM, but has 5 VCPUs, and is only 2x the price of an `m1.small` at 16.5c/hr.
+
+There currently no `c1.large`. Instead the `c1.xlarge` has 4x the RAM and CPUs of the `c1.medium` and is 4x the price.
+
+The High Memory family currently has three instance types - the `m2.xlarge`, `m2.2xlarge` and `m2.4xlarge`. Each has 2x the CPU, RAM, I/O and price attributes of the former. The `m2.xlarge` offers 17.1G of RAM, only 6.5 VCPUs and costs 45c/hr.
+
+If I/O is your job or entire environments primary bottleneck, then there is the Cluster Compute family of instance types.
+
+In each family of instance types, the largest instance type has the highest I/O. To understand why, imagine how AWS is creating instance types. They take physical hardware - motherboards, CPUs, RAM and an ethernet cable - and virtualize it. The smaller the instance types, the more VMs that the hardware is supporting and the smaller the available I/O is to each VM. If you get the largest VM in a family, then you're getting all the available I/O (probably about 1Gbit) and are not sharing it with anyone.
+
+## Scaling with BOSH
+
+Now that we know all about AWS instance types, let's change our deployment.
+
+* nginx: 1 x m1.small - that should be fine for our evented web server; as traffic scales it might want to grow to gain more I/O
+* wordpress: 1 x m1.small to 3 x m1.small - we can scale our wordpress jobs by adding more small instances
+* mysql: 1 x m1.small to 1 x m1.xlarge - give our SQL database a healthy combination of high I/O, high RAM and CPU.
+
+That is, we're going to add 2 m1.smalls for wordpress, and upgrade the mysql job to a larger instance type.
+
+This is easy with BOSH. We'll change our `wordpress-aws.yml` deployment manifest, re-run `bosh deploy` and the BOSH director orchestrates everything - from detaching and reattaching disk volumes (to keep our MySQL data safe) to deprovisioning and provisioning new VMs and their static IP addresses.
+
+Make the following changes to your `wordpress-aws.yml` deployment manifest for the `mysql` job:
+
+Add a new `resource_pool` for the special mysql instance:
+
+```yaml
+resource_pools:
+ - name: common
+ ...
+ - name: mysql
+ network: default
+ size: 1
+ stemcell:
+ name: bosh-stemcell
+ version: 0.5.1
+ cloud_properties:
+ disk: 8192
+ instance_type: m1.xlarge
+ availability_zone:
+ key_name:
+```
+
+Change the `mysql` job to use this new `resource_pool: mysql`. Note that the explicit `instance_type: m1.xlarge` changes too.
+
+```yaml
+jobs:
+...
+ - name: mysql
+ template: mysql
+ instances: 1
+ resource_pool: mysql
+ persistent_disk: 16384
+ networks:
+ - name: default
+ default: [dns, gateway]
+ cloud_properties:
+ instance_type: m1.xlarge
+```
+
+Make the following changes for the `wordpress` job. We no longer need a `m1.small` for `mysql`, and need 1 for `nginx` and 3 for `wordpress`, which is 4 `m1.smalls` in total:
+
+```yaml
+resource_pools:
+ - name: common
+ network: default
+ size: 4
+```
+
+Next, allocate 3 instances (VMs) to `wordpress`.
+```yaml
+jobs:
+...
+ - name: wordpress
+ template: wordpress
+ instances: 3
+```
+
+Run `bosh deploy` to confirm the differences and apply the changes:
+
+```
+$ bosh deploy
+
+```
Please sign in to comment.
Something went wrong with that request. Please try again.