Plodo is a tool I built to build and deploy Podiant's web and worker servers, in an immutable fashion, with new servers being spun up to replace old ones, instead of constantly updating existing servers (or "cattle" vs "pets").
The process for creating images and deploying droplets isn't quick, because DigitalOcean still takes a bit of time to create droplets from images, but it works for me.
This tool is designed to run with Python 3, and is not tested against Python 2.
Simply run:
$ pip install plodo
or
$ pip3 install plodo
The tool is designed to be run directly from the terminal, so doesn't need to be in a virtualenv.
When installed, Plodo will look for a configuration file (called plodo.conf) in the current working directory. A smaple configuration file looks like this:
---
digitalocean:
api_version: 2
api_token: abcdefghijklmnopqrstuvwxyz1234567890 # Your DigitalOcean API token
region: lon1 # The DigitalOcean region to create droplets and snapshots
ssh_keys:
12345678: /path/to/keyfile # The ID of an SSH key stored within DigitalOcean, and the local path to the key file
ansible:
playbook: ./ansible/production.yaml # The Ansible playbook to run
home: /var/app/ # The remote home directory for the app
racks:
production: # The default production rack to deploy to
images: # A collection of arbitrary group names with configuration options
cron:
size: 512mb # The droplet size
base: ubuntu-16-04-x64 # The base image to use if running `plodo build` for the first time
monitoring: true # Enable monitoring for the droplet
worker:
size: 2gb
base: ubuntu-16-04-x64
monitoring: true
web:
size: 2gb
base: ubuntu-16-04-x64
monitoring: true
front: true # Denotes a public-facing droplet, which changes the order of how droplets are destroyed and created
droplets: # The target number of droplets to deploy in each group
cron: 1
worker: 2
web: 5
load_balancer:
id: 725498b1-6553-4e88-9d14-e8812f9f591d # The ID of a DigitalOcean load balancer to add droplets to
group: web # The group whose droplets should be added to the load balancer
ansible: # Overrides the Ansible options for this rack
...
staging: # Further optional racks
...
shell:
group: worker # The droplet group used to find servers that can run a shell command
user: root # The Unix user to SSH in as
command: podiant shell # The command to run on the droplet to access a shellYou should store the file in the root directory of your app, and run plodo from there.
If you want to add the file to your repo, you can change the configuration accordingly:
# Remove the `digitalocean.api_key` setting, or the entire `digitalocean` object.
digitalocean:
api_version: 2
# Change `ssh_keys` from an object to a list of DigitalOcean SSH key IDs
ssh_keys:
- 12345678Once done, you can set the following environment variables:
SSH_KEY_12345678=/path/to/sshkey # Change '12345678' to the ID of your SSH key within DigitalOcean
DO_API_VERSION=2 # Optional
DO_API_TOKEN=abcdefghijklmnopqrstuvwxyz1234567890 # Your DigitalOcean API tokenTo use it:
$ plodo --help
You can add the --help option to any command to display a list of options and arguments for that command.
To diagnose issues, use the -d or --debug option. This will show a complete stack trace of errors, when raised.
This command creates either completely fresh snapshots - based off an Ubuntu image - or layer changes ontop of an existing snapshot. A droplet is spun up, an Ansible playbook is run, then the state of the droplet is saved as a snapshot.
The next time plodo build is run, it sues that snapshot to create a new one, then removes the old one.
Once the snapshots are saved, the droplets used to create them are destroyed.
Optionally specify a list of droplet groups (such as web or worker) that should be built. This is useful if you only need to rebuild a worker server's snapshot, for example.
-n,--no-destroy: Do not destroy droplets after using them to create snapshots.
This command spins up new droplets using the snapshots created in plodo build. It runs an Ansible playbook to deliver updated files, start necessary services, and add web droplets to a load balancer.
Once the new droplets are in place, old droplets are removed (this is done in a different order depending on whether the servers are public-facing or not).
Optionally specify a list of droplet groups (such as web or worker) that should be deployed. This is only really useful in situations where you quickly need to upload new code to a specific server group, as the end goal is to have all server groups running the same code.
This command runs the Ansible playbook against a given DigitalOcean droplet. It's really a shortcut for quickly deploying code changes to a specific droplet, and was written to test and diagnose provisioning of droplets via Ansible.
The first argument must be an Ansible tag. This tag is added to the ansible-playbook command.
The second argument is the ID to a DigitalOcean droplet.
This command is used to scale a group of servers up or down. Plodo will then ensure the specified number of droplets are running, and either remove droplets no longer needed, or add new ones.
Specify the group name and a number in key-pair fashion, like so:
$ plodo scale web=6 worker=2This command uses SSH to run an arbitrary command on the first server in a group, specified in the config.
Because the tool was initially designed to deploy a Django app, my config uses an equivalent of manage.py shell to access the Django shell, so I can interact with models with one command, without ever needing to know the IP address of a server.
This allows for the viewing or following of log files from a given group, or number of groups.
Specify each group by name. The command runs journalctl (optionally with -f) on the necessary droplets, and pipes the output to the local console.
-t,--tail: Tail (or follow) the logs, otherwise the last 100 lines will be printed.
-
plodo scalehas an issue where it incorrectly calculates the number of droplets it needs to create, when scaling to 1. -
plodo logsis very primitive and doesn't exit cleanly. It just instantiates n threads, where n is the number of servers in each group, so it can simultaneously SSH into each server and pipe the output to the local console. This is not sustainable, but was hacked together quickly because I needed to see some logs and I was too cheap to increase my Papertrail storage.
I can't guarantee that I'll be able to provide support, but if you notice a bug or you have an improvement, I'll gladly accept a pull request.
The tool was built quickly with a singular purpose, and was tested IRL. I am a fan of TDD but it wasn't an appropriate methodology for the work I needed to do here.
If you'd like to write tests, perhaps by faking responses from the DigitalOcean API, I'll be more than happy to accept a pull request.