winsible is an experimental wrapper for ansible-playbook
that enhances its multiprocessing and connection persistence, for platforms where OpenSSH can't transparently do its own connection pooling (such as EL5 and Cygwin).
And unlike Ansible's own accelerate
or fireball
modes, it doesn't require installing anything on the target computers: you just install it on the machine where you run your playbooks, and use winsible-playbook
instead of ansible-playbook
.
Install winsible with pip install winsible
or easy_install winsible
. If you don't have these commands on your system, try installing your platform's python-setuptools
package. (Also, if you're not on Cygwin, you'll need to be root or use sudo, or install them to a virtualenv.)
If you have a current version of gevent
on your system (1.0.1 or better), winsible can use it to eliminate forking altogether and run plays against all hosts with a single process. But this is a relatively small performance boost compared to connection pooling (and is somewhat experimental), so it's optional and has to be explicitly activated. If you want to try it, you can install gevent with sudo pip install "gevent>=1.0.1"
or sudo easy_install "gevent>=1.0.1"
. (Then enable it with ANSIBLE_PROCESS_MODE=gevent
in the environment or process_mode=gevent
in your ansible.cfg's [defaults]
section.)
Last, but not least, make sure you've got an ssh
command: if you're on Cygwin, install the openssh
package, or on Linux distros it might be in an openssh-clients
package.
Once you have these things installed, you're ready to use winsible-playbook
in place of ansible-playbook
, to enjoy faster playbook runs. (A winsible
script is also included, for completeness, but it doesn't provide much in the way of acceleration. It does, however, tweak ansible's defaults so that the ssh
transport will work correctly on Cygwin.)
Note: At the moment, this project as a whole is still pretty alpha. (Especially the gevent
mode.) In general, winsible has not been run on a very wide variety of hosts, modules, or plugins, so experience reports are welcome. If something breaks, or if it doesn't improve performance in your use case, please file an issue on the github page with some details so I can have a look at it. Thanks!
The winsible
and winsible-playbook
scripts invoke ansible with a preselected "processing mode", which can be one of fork
, gevent
, pool
, or smart
. This mode can be configured using the process_mode
variable in the [defaults]
section of your ansible.cfg, or via the ANSIBLE_PROCESS_MODE
environment variable. (The default mode is smart
, which will use pool
if running a playbook, and fork
otherwise.)
The fork
mode is Ansible's standard way of multiprocessing, which opens connections in separate processes. This works fine if you're running a single task or have a ControlPersist
-capable ssh, but is terribly inefficient otherwise.
So the gevent
mode patches Ansible to use a gevent task pool instead of separate processes, so that host connections can be shared (and reused) within a single process. The pool
mode works similarly, but uses a multiprocessing.SyncManager
to create a dedicated pool process, which the Ansible-forked task workers talk to via IPC. It can be less efficient than the gevent
mode (due to inter-process communication overhead and locking) but it's a lot less invasive and thus more likely to be compatible with existing plugins, transports, and modules that do any of their own I/O (and whose behavior might thus be affected by gevent's monkeypatching).
While the gevent
mode will let an entire playbook run against multiple hosts in parallel without forking the main process even once (instead of once per host per task), the primary performance benefit of winsible comes from its ability to pool SSH connections and reuse them over the lifetime of a playbook run.
Both the gevent
and pool
modes support this connection pooling; they just differ as to which process the connection pool lives in. (In gevent
, connections are pooled in the main process, while the pool
mode creates a dedicated multiprocessing.Manager
process that holds the pool.) Even Ansible's standard fork
mode can benefit from connection pooling, in the case of a playbook being run against a single host, or with the Ansible forks
option set to 1, since such tasks are run in a single process.
In order to take advantage of this feature, however, you have to use a connection type that supports pooling. Unfortunately, Ansible's built-in transports don't do this, unless you're using ssh on a version and platform that supports ControlPersist. (Something which is probably never going to be available on Cygwin.)
So winsible includes three connection-pooling transports: paramiko_pool
(cross-platform) and plink
(Cygwin-specific). These transports are specifically designed to work with winsible's pool
and gevent
modes, but they can also speed up Ansible's standard fork
mode for single-host plays or with forks=1
. (On Cygwin, the plink
transport is also faster than Paramiko, due to being written in C and using the Windows API directly rather than via Cygwin translation. But it's not as fast as native ssh
if you're running a single task rather than a playbook.)
Whenever you use the gevent
or pool
modes, winsible remaps Ansible's ssh
and paramiko
transports to use its own replacements. If you need to override this for some reason (e.g. debugging), Ansible's built-in transports can be accessed using the names _ssh
and _paramiko
in the appropriate config files, command-line options, environment variables, or playbook settings.
The included .exe
files (for the Cygwin-specific plink
transport) are from Jakub Kotrla's registry-free version of PuTTY; see the PUTTY_LICENSE file in the winsible/
directory for the full credits. (Basically, though, it's MIT-licensed, and copyright 1997-2015 Simon Tatham, with a long list of additional contributors.)
The rest of Winsible is copyright 2015 PJ Eby, and MIT-licensed as follows:
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT OWNERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.