simple controller for parallel script execution
Clojure Go CoffeeScript HTML Makefile
Latest commit 6128842 Oct 27, 2016 @pyr project: bump clojurescript

README.md

warp: distributed workflow management

warp distributes scenarios across any number of machines.

hubot: deploy to production, engage!

build status

The story

Your applications span a large group of hosts and deployment involves several separate steps.

Let's say that you have a web application deployment process that involves:

  • Updating a git repository
  • Restarting a service

This process is likely the same for several profiles such as test, staging and production.

warp scenarios

Warp provides a DSL for writing scenarios and schedules executions over a pub-sub system, streaming the results to the controller which makes results available through an API and web view

Command executions can be scheduled through the following means:

  • The web interface exposed by the controller
  • API queries to the controller
  • IRC/Campfire/Hipchat through a hubot script

A sample scenario

{:timeout 2
 :name    "ping"
 :matcher {:type :none}
 :profiles {:everyone {:type :all}
            :platform {:type :and :clauses [{:type :fact :fact "facter.sp_environment" :value "{{0}}"}
                                            {:type :fact :fact "facter.platform" :value "{{1}}"}]}
            :prod     {:type :fact :fact "facter.sp_environment" :value "prod"}
            :preprod  {:type :fact :fact "facter.sp_environment" :value "preprod"}
            :host     {:type :host :host "{{0}}"}}
 :commands [{:type :ping}]}

More screenshots

warp index warp output

Pub-Sub support

Scenario execution happens through an HTTP api.

Warp and warp-agent borrow from mcollective and my first implementation within amiral

Warp aims to improve on amiral in the following ways:

  • Bundle matchers, timeouts and a list of commands (a script) options in named "scenarios"
  • Extract out of the IRC bot framework and display extended execution results in a web view
  • Deprecate signing requests with ssh-keys and move to SSL
  • Provide a lighter-weight agent
  • Support arguments

Development

To build and run the controller:

lein cljsbuild
lein run -- -f doc/controller.clj

When working on the ClojureScript part, you can automatically rebuild it when a change happens:

lein cljsbuild auto

You can run the agent with:

./agent/warp-agent doc/warp-agent.json