Providers
While rubber has its roots in AWS, it can support multiple providers for various cloud services. This page lists each of the supported providers.
A computing provider corresponds to a server that you deploy your application to. E.g., EC2 is Amazon's computing provider, part of the larger AWS suite of services.
Currently the same computing provider must be used for all nodes in a cluster. However, you can use different providers in different clusters. One way to achieve this is with environment overrides in your configuration. For example:
# config/rubber/rubber.yml
cloud_provider: aws
# config/rubber/rubber-staging-env.yml
cloud_provider: digital_ocean
This will use EC2 by default for all environments, except the staging environment, where DigitalOcean will be used instead.
NB: While we try to maintain a common configuration for each provider, the values are going to be provider specific. E.g., image_type can be use with both aws and digital_ocean, but Amazon and DigitalOcean have their own image types with their own unique identifiers. You may need to consult the API documentation for the provider you'd like to use to see the domain of acceptable values.
EC2 is the cloud computing portion of Amazon's AWS suite of services. It's enabled by using:
cloud_provider: aws
Its required configuration is:
cloud_providers:
aws:
access_key: "YOUR_AWS_ACCESS_KEY"
secret_access_key: "YOUR_AWS_SECRET_ACCESS_KEY"
account: 123456789
key_name: your_ssh_keypair_name
key_file: "#{Dir[(File.expand_path('~') rescue '/root') + '/.rubber_your_app/*' + cloud_providers.aws.key_name + '.pem'].first}"
pk_file: "#{Dir[(File.expand_path('~') rescue '/root') + '/.rubber_your_app/pk-*'].first}"
cert_file: "#{Dir[(File.expand_path('~') rescue '/root') + '/.rubber_your_app/cert-*'].first}"
The EC2 provider has optional configuration items as well, each of which may be overridden on a per-host basis, including region, availability_zone, image_type, and image_id. Please see the sample rubber.yml for an annotated list of example attributes.
DigitalOcean is a cloud computing provider that uses SSD-backed instances and offers a wide arrange of server configuration types. It does not offer a full suite of services like AWS, but it can be substantially cheaper and faster than EC2 for certain server loads.
It's enabled by using:
cloud_provider: digital_ocean
Its required configuration is:
cloud_providers:
digital_ocean:
client_key: your_digital_ocean_client_key
api_key: your_digital_ocean_api_key
key_name: your_ssh_key_name
key_dir: "#{(File.expand_path('~') rescue '/root') + '/.rubber_your_app' }"
key_file: "#{Dir[key_dir + '/*' + key_name].first}"
NB: You do not need to have your SSH key created in Digital Ocean first. Part of the server creation process on rubber's end is to create the key if necessary. Although if you do have the key already created in your account, that's fine too. In both cases you do need to have the private and public key files stored on your local machine so rubber can access them.
The DigitalOcean provider has optional configuration items as well, each of which may be overridden on a per-host basis, including region, image_type, and image_id. If you're coming from EC2, note that there is currently no equivalent to availability_zone for DigitalOcean: each region is its own data center. Please see the sample rubber.yml for an annotated list of example attributes.
Finally, note that DigitalOcean does not have an account-wide firewall like EC2's security groups. As a result, Rubber reimplements security groups as a set of iptables rules on your DigitalOcean server. This means you can still set up rules declaratively in your Rubber configuration. But, the corollary is any changes you make manually to your iptables configuration on a server is likely to be lost. Rubber cannot merge existing iptables rules with its own and thus expects to manage all of them.
If you'd like to help the rubber project out, please use our referral link when signing up for DigitalOcean. The referral amounts aren't large, but they help us cover the costs of running the servers necessary to further develop the DigitalOcean support.
Sometimes you need to use rubber with a provider that doesn't offer an API (e.g., using dedicated hardware) or an API that rubber doesn't currently support. In these cases, you can use our generic provider. Naturally you can't use the server management facilities in rubber, but you can deploy and manage your role-based configurations. You'll also be able to interface with non-computer providers, so your backups will still push to S3 if you have that configured.
In order to support generic server configurations, Rubber needs to make some assumptions about your computing environment. If your environment doesn't match these criteria there's a very good chance this provider simply won't work. The prerequisites are:
- You must be running Ubuntu (Debian might work, but it's untested).
- You must be running an SSH server.
- Your SSH server must allow root logins (key-based only).
- Your user account must have sudo access.
- Your user account must accept key-based SSH authentication using your configured key file.
You can enable this provider with:
cloud_provider: generic
Its required configuration is:
cloud_providers:
generic:
key_file: "#{Dir[(File.expand_path('~') rescue '/root') + '/.rubber_your_app/*.pem'].first}"
You will also need to set the Capistrano :initial_ssh_user value to be the name of your user account with sudo access. Keep in mind this account must be able to do key-based authentication with the configured key file. You can configure Capistrano like so:
# config/deploy.rb
set :initial_ssh_user, 'your_user'
Finally, note that once set up, this server will be configured similarly to other providers that Rubber interfaces with. Namely, your root account will be enabled and can be logged into remotely using the configured key file. Also, Rubber manages its security groups configuration through iptables rules. You should no longer use iptables directly, but rather set up security group configuration rules in your rubber configuration. Rubber cannot merge security group rules and thus expects to manage all of them.
Vagrant is a great tool for automating the creation of virtual machines. It is most often used for testing out development or staging environments. Rubber ships with a Vagrant plugin that allows it to be used as a Vagrant provisioner. Used in tandem, they allow you to quickly bring up and bootstrap a virtual machine to test out your Rubber configuration and your application.
You can enable this provider with:
cloud_provider: vagrant
Vagrant has no required configuration, but as of Rubber 2.5.0 there's a quirk of Rubber where it expects a cloud provider to have configuration. In order to satisfy Rubber until that's resolved, you'll need to specify a dummy configuration:
cloud_providers:
vagrant:
# Rubber assumes every cloud provider has configuration. Vagrant really doesn't need anything, but until the core
# of Rubber handles that, just set a dummy value.
dummy: true
The vagrant cloud provider is a specialization of the generic provider, where we already know the instance's IP configuration. Please review the docs on the generic provider for any caveats you should be aware of.
vSphere is the commercially licensed version of VMware's ESX server, which allows for programmatic control of a cluster of virtual machines. It allows you to create your own private computing cloud with VMs of arbitrary size on your own hardware with automatic failover. If you're looking for a hosted vSphere solution, MacStadium has very good prices and a great sales & support team. (Please note that no one on the rubber project makes any money from endorsing MacStadium -- we've just been very pleased with their quality of service and pricing).
You can enabled this provider with:
cloud_provider: vsphere
Its required configuration is:
cloud_providers:
vsphere:
vcenter_host: your_vcenter_hostname_or_ip_address
vcenter_username: your_vcenter_username
vcenter_password: your_vcenter_password
expected_pubkey_hash: your_vsphere_api_public_key_hash
key_name: your_ssh_key_name
key_dir: "#{(File.expand_path('~') rescue '/root') + '/.rubber_your_app' }"
key_file: "#{Dir[key_dir + '/*' + key_name].first}"
The only configuration item you probably don't know off-hand is the public key hash. I have no idea why this is required because when you supply the wrong one, the correct one is printed out for you. So, if you don't know the value, just run with the provider once and extract it from the error message when the Rubber process fails.
Unlike other cloud providers, the vSphere provider can create VMs of arbitrary size. Likewise, the VMs do not come with a pre-configured IP address, so you'll need to configure that. Both of these require Rubber configuration you may have not seen before:
hosts:
staging:
cloud_providers:
vsphere:
region: 'Atlanta' # The name of the datacenter in vSphere.
image_id: 'base_bootstrapped-12.10-2013_10_31' # The name of the VM or template to clone.
public_network_name: 'Internet Network' # The name of the virtual switch for the public network.
private_network_name: 'Private Network' # The name of the virtual switch for the private network.
memory: 8192 # Memory in MBs.
cpus: 8 # Number of single-core CPUs.
public_nic:
ip_address: '127.0.0.1'
subnet_mask: '255.255.255.240'
gateway: '127.0.0.1'
dns_servers: ['8.8.8.8', '8.8.4.4']
private_nic:
ip_address: '192.168.0.1
subnet_mask: '255.255.255.0'
dns_servers: ['8.8.8.8', '8.8.4.4']
volumes:
- size: 100
device: /dev/sdb
mount: /mnt
filesystem: ext3
thin: false # Whether to create the virtual disk as thin or thick.
eager_zero: false # Whether to create a thick virtual disk as eager zero or lazy zero.
NB: Rubber needs to clone an existing VM or template to create hosts for you. This VM must have VMware Tools installed. You will need to create this base VM manually or find one in one of the public VM repositories, but this is a hard requirement of using vSphere with Rubber.
In order to avoid duplication in each of your host configurations, you can define fallback defaults at the cloud-level similar to other Rubber configurations. E.g.,
cloud_providers:
vsphere:
region: 'Atlanta' # The name of the datacenter in vSphere.
image_id: 'base_bootstrapped-12.10-2013_10_31' # The name of the VM or template to clone.
public_network_name: 'Internet Network' # The name of the virtual switch for the public network.
private_network_name: 'Private Network' # The name of the virtual switch for the private network.
public_nic:
subnet_mask: '255.255.255.240'
gateway: '127.0.0.1'
dns_servers: ['8.8.8.8', '8.8.4.4']
private_nic:
subnet_mask: '255.255.255.0'
dns_servers: ['8.8.8.8', '8.8.4.4']
hosts:
staging:
memory: 8192 # Memory in MBs.
cpus: 8 # Number of single-core CPUs.
public_nic:
ip_address: '127.0.0.1'
private_nic:
ip_address: '192.168.0.1
volumes:
- size: 100
device: /dev/sdb
mount: /mnt
filesystem: ext3
thin: false # Whether to create the virtual disk as thin or thick.
eager_zero: false # Whether to create a thick virtual disk as eager zero or lazy zero.
And finally, if you find that you need to override the cloud-level config for the gateway value on a per-host basis, you can do so by setting the value to null:
hosts:
staging:
private_nic:
ip_address: '192.168.0.1
gateway: null
vSphere provides a large amount of flexibility in creating your cluster. While the volumes configuration in the previous configuration examples is strictly not necessary, we recommend that you start with a base VM with a small root store (<= 20 GB). You can use this base VM for all hosts in your cluster and then configure your storage needs by mounting additional virtual disks (volumes). The alternative is to create a variety of VM templates with various root store sizes. There is little gain to doing so and it increases management overhead for you rather substantially.
For those familiar with EC2, this recommendation roughly follows the EC2 model of a small root store with a large ephemeral store attached and mounted to /mnt.
A storage provider corresponds to a remote data store that you read and write files or objects from. E.g., S3 is Amazon's storage provider, part of the larger AWS suite of services.
Rubber uses the storage provider for various utility operations. The most common case is storing off-server backups. The storage provider is optional, but not having it configured will disable certain Rubber features.
S3 is the cloud computing portion of Amazon's AWS suite of services. It's enabled simply by configuring your AWS credentials. If you're already using the EC2 compute provider, there is no additional work necessary. Otherwise, your configuration should look like:
cloud_providers:
aws:
access_key: "YOUR_AWS_ACCESS_KEY"
secret_access_key: "YOUR_AWS_SECRET_ACCESS_KEY"
account: 123456789
A DNS provider is used for managing host records from within Rubber. Rubber makes use of fog for its supported DNS providers and thus theoretically all providers fog supports, rubber equally supports. In practice, sometimes a provider needs additional work to be fully supported in rubber. If you come across one, please file an issue.
Route53 is the DNS portion of Amazon's AWS suite of services. It's enabled with:
dns_provider: aws
Its required configuration is:
dns_providers:
aws:
access_key: "#{cloud_providers.aws.access_key}"
access_secret: "#{cloud_providers.aws.secret_access_key}"
As a convenience, the above configuration references the already configured values assuming you're already using EC2 or S3. If you're not using either service, you'll need to update the above values with your AWS access key and AWS secret access key.
Zerigo is a cloud computing company that had its start with its flagship DNS service. It's enabled with:
dns_provider: zerigo
Its required configuration is:
dns_providers:
zerigo:
email: your_zerigo_account_email_address
token: your_zerigo_api_token
DynDNS is a premium DNS service that offers many performance benefits over other providers. It's enabled with:
dns_provider: dyndns
Its required configuration is:
dns_providers:
dyndns:
user: your_dyndns_username
password: your_dyndns_password
There is one optional configuration item, as well: the update_url, which defaults to the template URL https://members.dyndns.org/nic/update?hostname=%host%&myip=%ip%.
Nettica provides a low latency DNS service with fast updates. It's enabled with:
dns_provider: nettica
Its required configuration is:
dns_providers:
nettica:
user: your_nettica_username
password: your_nettica_password