A collection of Ruby daemons flying in close formation. AKA our startling message-based deploy environment.
Weblog-based post-hack rationalisation: http://ops.failcake.net/blog/2012/04/19/one-very-important-thought/
More of the same, containing example Jenkins-plumbing: http://ops.failcake.net/blog/2012/04/19/a-hazelnut-in-every-bite/
One-click (more or less. Work with me here...) deploys. Jenkins integration. Standard bus. Devops-meccano.
Some diagrams of how it works in practice for us.
Developer commits a change to the master git repo, and a post-commit-hook triggers a message to the queue:
Jenkins receives the message and runs the appropriate tests, then sends a success or failure message to the queue:
The success message is received by all the servers that configuration management has told that they should have that code on them:
Servers git fetch the latest changes, so that they always have a local copy of the full repo, to reduce dependencies on the central git server for deploys:
Deployment works as follows:
User uses a lightweight web wrapper to select what tag they want to deploy and to what servers, MCollective client issues a message to the queue:
MCollective agent receives the message, triggers the deploy process, running pre and post deploy scripts:
Other servers deploy if this succeeds, orchestrated by MCollective:
Trigger repo updates from Github/Gitlab
Bonus: 'Manage' (ht)access remotely
Stomp-jenkins: Watch future.git.commits, if a project we care about has been updated, prod Jenkins via HTTP to kick off a build. (future.git.commits is one of our own topics. Yours should be different.)
Stomp-repo: Watch for package-update messages emitted by the Debian package repository. Run apt-get update if a package we're interested in has a new version. (Not quite enough blind trust to install the thing automatically. That's what puppet/mcollective is for.)
Jenkins-helper: Emit status messages on future.jenkins and future.events.jenkins to the effect that a job has started, failed or succeeded. (A better way of doing this would be a STOMP plugin for Jenkins, obviously.)
puppet-module: Git repo management and stomp-git install/setup.
Server-malarkey: Hook scripts for git. Also hook script for SVN. I'm sorry.
Stomp-keepalive: Cheap and nasty stomp heartbeat emulator to keep firewalls happy. (STOMP 1.0 doesn't do heartbeats. They're in the 1.1 spec, but implemented in a wonky sort of way on the kit we use. So. This is a quick hack to emit heartbeat packets just often enough to stop our firewalls from dropping the connection between client and broker. We have a random sort of f/w config. Consider yourself lucky if you can't see the need for this.)