Permalink
Browse files

Reworked my Fabric fork as a standalone package.

  • Loading branch information...
0 parents commit c7755e71cfa2881151be1546d485ffdbe84fa483 @tav committed May 16, 2011
Showing with 3,519 additions and 0 deletions.
  1. +72 −0 .gitignore
  2. +19 −0 AUTHORS
  3. +3 −0 MANIFEST.in
  4. +51 −0 README.rst
  5. +206 −0 UNLICENSE
  6. +19 −0 bolt/__init__.py
  7. +9 −0 bolt/api.py
  8. +1,564 −0 bolt/core.py
  9. +1,516 −0 bolt/fabcode.py
  10. +60 −0 setup.py
72 .gitignore
@@ -0,0 +1,72 @@
+# specific files/directories
+
+build
+dist
+bolt.egg-info
+
+# minified css/js files
+
+*.min.js
+*.min.css
+
+# hidden files/directories
+
+.bzr
+.bzrignore
+.DS_Store
+.hg
+.hgignore
+.lock-wscript
+.sass-cache
+.sconsign*
+.svn
+
+# file patterns
+
+*#*
+*~
+*.5
+*.6
+*.8
+*.a
+*.dylib
+*.egg
+*.jar
+*.la
+*.lo
+*.o
+*.out
+*.pyc
+*.pyo
+*.so
+*.swp
+*.tar.bz2
+*.tar.gz
+*.tbz
+*.tbz2
+*.tgz
+*.rdb
+
+_testmain.go
+
+# file patterns (xcode)
+
+*.mode1v3
+*.mode2v3
+*~.nib
+*.pbxuser
+*.perspective
+*.perspectivev3
+*.swp
+*.tm_build_errors
+
+# file patterns (windows)
+
+*.dll
+*.exe
+*.ilk
+*.lib
+*.ncb
+*.pdb
+*.suo
+*.vcproj.*.*.user
19 AUTHORS
@@ -0,0 +1,19 @@
+Bolt Authors
+============
+
+This is the official list of the Bolt Authors ("The Authors"), listed in
+alphabetical order:
+
++----------------------------+--------------------+--------------------------+----------------+
+| Name | Nick | Email | Location |
++============================+====================+==========================+================+
+| `Tav`_ | tav | tav@espians.com | U.K. |
++----------------------------+--------------------+--------------------------+----------------+
+
+.. Please keep the listing in Alphabetical Order, thanks!
+
+.. _Tav: http://tav.espians.com
+
+By adding yourself to this list, you explicitly agree to affirm all of your
+Contributions to Bolt ("The Work") to be covered by the Unlicense found in the
+UNLICENSE file.
3 MANIFEST.in
@@ -0,0 +1,3 @@
+include AUTHORS
+include UNLICENSE
+include README.rst
51 README.rst
@@ -0,0 +1,51 @@
+Bolt lets you easily automate sysadmin tasks like deployment. You can use it to
+manage multi-server setups over SSH or even as a build tool. To use, simply
+create a ``Boltfile`` with your tasks, e.g.
+
+::
+
+ from bolt.api import *
+
+ @task
+ def deploy():
+ """publish the latest version of the app"""
+
+ with cd('/var/www/mysite.com'):
+ run('git remote update')
+ run('git checkout origin/master')
+
+ sudo("/etc/init.d/apache2 graceful")
+
+And then, run the tasks from the command line, e.g.
+
+::
+
+ $ bolt deploy
+
+Bolt was initially developed as a fork of `Fabric <http://fabfile.org/>`_, but
+has since been extracted as a standalone tool without any of the historic
+baggage of the Fabric APIs.
+
+**Documentation**
+
+Bolt doesn't currently have any docs, but you can look at the introduction to
+the Fabric fork for details of how to use most of its features. Simply replace
+the references to ``fab`` and ``fabric`` with ``bolt``:
+
+* `Fabric with Cleaner API and Parallel Deployment Support
+ <http://tav.espians.com/fabric-python-with-cleaner-api-and-parallel-deployment-support.html>`_
+
+**Contribute**
+
+To contribute any patches simply fork the repository on GitHub and send a pull
+request to https://github.com/tav, thanks!
+
+**License**
+
+The code derived from Fabric is contained within the ``bolt/fabcode.py`` file
+and is under the BSD license. The rest of the code has been released into the
+`Public Domain <https://github.com/tav/bolt/raw/master/UNLICENSE>`_. Do with it
+as you please.
+
+--
+Enjoy, tav <tav@espians.com>
206 UNLICENSE
@@ -0,0 +1,206 @@
+Unlicense
+=========
+
+.. contents:: Table of Contents
+ :depth: 1
+ :backlinks: none
+
+In the spirit of contributing to the Public Domain, to the full extent possible
+under law, the Bolt Authors ("The Authors") have waived all copyright, patent
+and related or neighboring rights to their Contributions to Bolt ("The Work").
+
+This does not apply to works authored by third parties ("Third Party Works")
+which come with their own copyright and licensing terms. These terms may be
+defined in explicit files within the `third_party` directories or specified as
+part of the contents of licensed files. We recommend you read them as their
+terms may differ from the terms below.
+
+All trademarks and registered trademarks mentioned in The Work are the property
+of their respective owners.
+
+
+Usage
+-----
+
+To affirm that a Contribution to The Work is covered by this Unlicense, add an
+informative header like::
+
+ # Public Domain (-) 2011 The Bolt Authors.
+ # See the Bolt UNLICENSE file for details.
+
+If the Contribution is to an existing Third Party Work, then it can be affirmed
+with an informative header like::
+
+ # Changes to this file by The Bolt Authors are in the Public Domain.
+ # See the Bolt UNLICENSE file for details.
+
+
+Statement of Purpose
+--------------------
+
+The laws of most jurisdictions throughout the world automatically confer
+exclusive Copyright and Related Rights (defined below) upon the creator and
+subsequent owner(s) (each and all, an "owner") of an original work of authorship
+and/or a database (each, a "Work").
+
+Certain owners wish to permanently relinquish those rights to a Work for the
+purpose of contributing to a commons of creative, cultural and scientific works
+("Commons") that the public can reliably and without fear of later claims of
+infringement build upon, modify, incorporate in other works, reuse and
+redistribute as freely as possible in any form whatsoever and for any purposes,
+including without limitation commercial purposes.
+
+These owners may contribute to the Commons to promote the ideal of a free
+culture and the further production of creative, cultural and scientific works,
+or to gain reputation or greater distribution for their Work in part through the
+use and efforts of others.
+
+For these and/or other purposes and motivations, and without any expectation of
+additional consideration or compensation, the person associating the Unlicense
+with a Work (the "Affirmer"), to the extent that he or she is an owner of
+Copyright and Related Rights in the Work, voluntarily elects to apply the
+Unlicense to the Work and publicly distribute the Work under its terms, with
+knowledge of his or her Copyright and Related Rights in the Work and the meaning
+and intended legal effect of the Unlicense on those rights.
+
+
+Definitions
+-----------
+
+The term "distribute" has the same meaning here as under U.S. copyright law. A
+"Contribution" is the original Work, or any additions or changes to it.
+
+A Work made available under the Unlicense may be protected by copyright and
+related or neighboring rights ("Copyright and Related Rights"). Copyright and
+Related Rights include, but are not limited to, the following:
+
+1. the right to reproduce, adapt, distribute, perform, display, communicate, and
+ translate a Work;
+
+2. moral rights retained by the original author(s) and/or performer(s);
+
+3. publicity and privacy rights pertaining to a person's image or likeness
+ depicted in a Work;
+
+4. rights protecting against unfair competition in regards to a Work, subject to
+ the Limitations and Disclaimers, below;
+
+5. rights protecting the extraction, dissemination, use and reuse of data in a
+ Work;
+
+6. database rights (such as those arising under Directive 96/9/EC of the
+ European Parliament and of the Council of 11 March 1996 on the legal
+ protection of databases, and under any national implementation thereof,
+ including any amended or successor version of such directive); and
+
+7. other similar, equivalent or corresponding rights throughout the world based
+ on applicable law or treaty, and any national implementations thereof.
+
+
+Waiver
+------
+
+To the greatest extent permitted by, but not in contravention of, applicable
+law, Affirmer hereby overtly, fully, permanently, irrevocably and
+unconditionally waives, abandons, and surrenders all of Affirmer's Copyright and
+Related Rights and associated claims and causes of action, whether now known or
+unknown (including existing as well as future claims and causes of action), in
+the Work (i) in all territories worldwide, (ii) for the maximum duration
+provided by applicable law or treaty (including future time extensions), (iii)
+in any current or future medium and for any number of copies, and (iv) for any
+purpose whatsoever, including without limitation commercial, advertising or
+promotional purposes (the "Waiver").
+
+Affirmer makes the Waiver for the benefit of each member of the public at large
+and to the detriment of Affirmer's heirs and successors, fully intending that
+such Waiver shall not be subject to revocation, rescission, cancellation,
+termination, or any other legal or equitable action to disrupt the quiet
+enjoyment of the Work by the public as contemplated by Affirmer's express
+Statement of Purpose.
+
+
+Public License Fallback
+-----------------------
+
+Should any part of the Waiver for any reason be judged legally invalid or
+ineffective under applicable law, then the Waiver shall be preserved to the
+maximum extent permitted taking into account Affirmer's express Statement of
+Purpose. In addition, to the extent the Waiver is so judged Affirmer hereby
+grants to each affected person a royalty-free, non transferable, non
+sublicensable, non exclusive, irrevocable and unconditional license to exercise
+Affirmer's Copyright and Related Rights in the Work (i) in all territories
+worldwide, (ii) for the maximum duration provided by applicable law or treaty
+(including future time extensions), (iii) in any current or future medium and
+for any number of copies, and (iv) for any purpose whatsoever, including without
+limitation commercial, advertising or promotional purposes (the "Public
+License").
+
+The Public License shall be deemed effective as of the date the Unlicense was
+applied by Affirmer to the Work. Should any part of the Public License for any
+reason be judged legally invalid or ineffective under applicable law, such
+partial invalidity or ineffectiveness shall not invalidate the remainder of the
+Public License, and in such case Affirmer hereby affirms that he or she will not
+(i) exercise any of his or her remaining Copyright and Related Rights in the
+Work or (ii) assert any associated claims and causes of action with respect to
+the Work, in either case contrary to Affirmer's express Statement of Purpose.
+
+
+Grant of Patent Rights
+----------------------
+
+Affirmer hereby grants to You a perpetual, worldwide, non-exclusive, no-charge,
+royalty-free, irrevocable (except as stated in this section) patent license to
+make, have made, use, offer to sell, sell, import, transfer and otherwise run,
+modify and propagate the contents of this Work, where such license applies only
+to those patent claims, both currently owned or controlled by Affirmer and
+acquired in the future, licensable by Affirmer that are necessarily infringed by
+this Work. This grant does not include claims that would be infringed only as a
+consequence of further modification of this implementation. If you or your agent
+or exclusive licensee institute or order or agree to the institution of patent
+litigation against any entity (including a cross-claim or counterclaim in a
+lawsuit) alleging that this Work or any Contribution incorporated within this
+Work constitutes direct or contributory patent infringement, or inducement of
+patent infringement, then any patent rights granted to you under this Grant of
+Patent Rights for the Work shall terminate as of the date such litigation is
+filed.
+
+
+Limitations and Disclaimers
+---------------------------
+
+1. No trademark rights held by Affirmer are waived, abandoned, surrendered,
+ licensed or otherwise affected by this document.
+
+2. Affirmer offers the Work as-is and makes no representations or warranties of
+ any kind concerning the Work, express, implied, statutory or otherwise,
+ including without limitation warranties of title, merchantability, fitness
+ for a particular purpose, non infringement, or the absence of latent or other
+ defects, accuracy, or the present or absence of errors, whether or not
+ discoverable, all to the greatest extent permissible under applicable law.
+
+ In no event shall the Affirmer be liable for any direct, indirect,
+ incidental, special, exemplary, or consequential damages (including, but not
+ limited to, procurement of substitute goods or services; loss of use, data,
+ or profits; or business interruption) however caused and on any theory of
+ liability, whether in contract, strict liability, or tort (including
+ negligence or otherwise) arising in any way out of the use of the Work, even
+ if advised of the possibility of such damage.
+
+3. Affirmer disclaims responsibility for clearing rights of other persons that
+ may apply to the Work or any use thereof, including without limitation any
+ person's Copyright and Related Rights in the Work. Further, Affirmer
+ disclaims responsibility for obtaining any necessary consents, permissions or
+ other rights required for any use of the Work.
+
+
+Appendix
+--------
+
+* The text of this document is derived from `Creative Commons CC0 1.0
+ Universal`_ and the `BSD style license`_ that ships with Google Go.
+
+* This Unlicense is seen as a mere transitional requirement until international
+ law adapts to the post intellectual property reality.
+
+.. _Creative Commons CC0 1.0 Universal: http://creativecommons.org/publicdomain/zero/1.0/legalcode
+.. _BSD style license: http://go.googlecode.com/hg/LICENSE
19 bolt/__init__.py
@@ -0,0 +1,19 @@
+# Public Domain (-) 2011 The Bolt Authors.
+# See the Bolt UNLICENSE file for details.
+
+"""
+====
+Bolt
+====
+
+Multi-server automation and deployment toolkit.
+
+::
+ _ _
+ | | | | _
+ | | _ ___ | || |_
+ | || \ / _ \ | || _)
+ | |_) )| |_| || || |__
+ |____/ \___/ |_| \___)
+
+"""
9 bolt/api.py
@@ -0,0 +1,9 @@
+# Public Domain (-) 2011 The Bolt Authors.
+# See the Bolt UNLICENSE file for details.
+
+from bolt.core import TIMEOUT, execute, failed, hook, shell, succeeded, task
+
+from bolt.fabcode import (
+ abort, cd, hide, get, env, fastprint, lcd, local, open_shell, output,
+ prefix, prompt, put, puts, reboot, run, settings, show, sudo, warn
+ )
1,564 bolt/core.py
@@ -0,0 +1,1564 @@
+# Public Domain (-) 2011 The Bolt Authors.
+# See the Bolt UNLICENSE file for details.
+
+"""Bolt Core."""
+
+import atexit
+import sys
+
+from cPickle import dumps, loads
+from fnmatch import fnmatch
+from imp import load_source
+from math import ceil
+from optparse import OptionParser
+from os import X_OK, access, chdir, environ, getcwd, listdir, pathsep, pipe
+from os.path import abspath, dirname, exists, expanduser, isabs, isdir, join
+from os.path import normpath, realpath, sep
+from random import sample
+from socket import AF_UNIX, SOCK_STREAM, error as socketerror, socket
+from struct import calcsize, pack, unpack
+from textwrap import dedent
+from time import time
+from traceback import format_exc
+from uuid import uuid4
+
+from Crypto.Random import atfork
+from tavutil.io import DEVNULL
+from tavutil.optcomplete import ListCompleter, autocomplete
+from yaml import safe_load as load_yaml
+
+from bolt.fabcode import (
+ HOST_REGEX, abort, AttrDict, blue, cyan, disconnect_all, _escape_split, env,
+ fastprint, _get_system_username, green, hide, indent, local, output, puts,
+ _rc_path, reboot, red, run, _setenv, settings, stringify_env_var, show,
+ sudo, yellow, warn
+ )
+
+try:
+ from errno import EAGAIN, EINTR, EPIPE
+ from os import close, fork, kill, read, remove, write
+ from signal import SIGALRM, SIGTERM, alarm, signal
+except ImportError:
+ forkable = False
+else:
+ forkable = True
+
+# ------------------------------------------------------------------------------
+# Some Constants
+# ------------------------------------------------------------------------------
+
+__version__ = "0.9"
+
+INDEX_HEADER_SIZE = calcsize('H')
+SHELL_BUILTINS = {}
+SHELL_HISTORY_FILE = None
+
+CACHE = {}
+DEFAULT = {}
+HOSTINFO = {}
+HOSTPATTERNINFO = {}
+HOSTPATTERNS = []
+
+# ------------------------------------------------------------------------------
+# Default Settings Loader
+# ------------------------------------------------------------------------------
+
+def get_settings(
+ contexts, env=env, cache=CACHE, default=DEFAULT, hostinfo=HOSTINFO,
+ hostpatterninfo=HOSTPATTERNINFO, hostpatterns=HOSTPATTERNS
+ ):
+ """Return a sequence of host/settings for the given contexts tuple."""
+
+ # Exit early for null contexts.
+ if not contexts:
+ return []
+
+ # Check the cache.
+ if contexts in cache:
+ return cache[contexts]
+
+ # Mimick @hosts-like behaviour when there's no env.config.
+ if 'config' not in env:
+ responses = []; out = responses.append
+ for host in contexts:
+ if host and (('.' in host) or (host == 'localhost')):
+ resp = {'host_string': host}
+ info = HOST_REGEX.match(host).groupdict()
+ resp['host'] = info['host']
+ resp['port'] = info['port'] or '22'
+ resp['user'] = info['user'] or env.get('user')
+ out(resp)
+ return cache.setdefault(contexts, responses)
+
+ # Save env.config to a local parameter to avoid repeated lookup.
+ config = env.config
+
+ # Set a marker to handle the first time.
+ if not cache:
+
+ cache['_init'] = 1
+
+ # Grab the root default settings.
+ if 'default' in config:
+ default.update(config.default)
+
+ # Grab any host specific settings.
+ if 'hostinfo' in config:
+ for host, info in config.hostinfo.items():
+ if ('*' in host) or ('?' in host) or ('[' in host):
+ hostpatterninfo[host] = info
+ else:
+ hostinfo[host] = info
+ if hostpatterninfo:
+ hostpatterns[:] = sorted(hostpatterninfo)
+
+ def get_host_info(context, init=None):
+ resp = default.copy()
+ if init:
+ resp.update(init)
+ info = HOST_REGEX.match(context).groupdict()
+ host = info['host']
+ for pattern in hostpatterns:
+ if fnmatch(host, pattern):
+ resp.update(hostpatterninfo[pattern])
+ if fnmatch(context, pattern):
+ resp.update(hostpatterninfo[pattern])
+ if host in hostinfo:
+ resp.update(hostinfo[host])
+ if context in hostinfo:
+ resp.update(hostinfo[context])
+ resp['host'] = host
+ resp['host_string'] = context
+ if info['port']:
+ resp['port'] = info['port']
+ elif 'port' not in resp:
+ resp['port'] = '22'
+ if info['user']:
+ resp['user'] = info['user']
+ elif 'user' not in resp:
+ resp['user'] = env.user
+ return resp
+
+ get_settings.get_host_info = get_host_info
+
+ else:
+ get_host_info = get_settings.get_host_info
+
+ # Loop through the contexts gathering host/settings.
+ responses = []; out = responses.append
+ for context in contexts:
+
+ # Handle composite contexts.
+ if '/' in context:
+ context, hosts = context.split('/', 1)
+ hosts = hosts.split(',')
+ base = config[context].copy()
+ additional = {}
+ for _host in base.pop('hosts', []):
+ if isinstance(_host, dict):
+ _host, _additional = _host.items()[0]
+ additional[_host] = _additional
+ for host in hosts:
+ if host in additional:
+ resp = get_host_info(host, base)
+ resp.update(additional[host])
+ out(resp)
+ else:
+ out(get_host_info(host, base))
+
+ # Handle hosts.
+ elif ('.' in context) or (context == 'localhost'):
+ out(get_host_info(context))
+
+ else:
+ base = config[context].copy()
+ hosts = base.pop('hosts')
+ for host in hosts:
+ if isinstance(host, basestring):
+ out(get_host_info(host, base))
+ else:
+ if len(host) > 1:
+ raise ValueError(
+ "More than 1 host found in config:\n\n%r\n"
+ % host.items()
+ )
+ host, additional = host.items()[0]
+ resp = get_host_info(host, base)
+ resp.update(additional)
+ out(resp)
+
+ return cache.setdefault(contexts, responses)
+
+# ------------------------------------------------------------------------------
+# Auto Environment Variables Support
+# ------------------------------------------------------------------------------
+
+class EnvManager(object):
+ """Generator for environment variables-related context managers."""
+
+ cache = {}
+
+ def __init__(self, var):
+ self.var = var
+
+ @classmethod
+ def for_var(klass, var):
+ cache = klass.cache
+ if var in cache:
+ return cache[var]
+ return cache.setdefault(var, klass(var))
+
+ def __str__(self):
+ return stringify_env_var(self.var)
+
+ def __call__(
+ self, value=None, behaviour='append', sep=pathsep, reset=False,
+ _valid=frozenset(['append', 'prepend', 'replace'])
+ ):
+ if value is None:
+ return stringify_env_var(self.var)
+ if behaviour not in _valid:
+ raise ValueError("Unknown behaviour: %s" % behaviour)
+ key = '$%s' % self.var
+ val = []
+ if (not reset) and (behaviour != 'replace'):
+ if key in env:
+ val.extend(env[key])
+ val.append((value, behaviour, sep))
+ kwargs = {key: tuple(val)}
+ return _setenv(**kwargs)
+
+# ------------------------------------------------------------------------------
+# Hooks Support
+# ------------------------------------------------------------------------------
+
+HOOKS = {}
+DISABLED_HOOKS = []
+ENABLED_HOOKS = []
+
+def hook(*names):
+ def register(func):
+ for name in names:
+ name = name.replace('_', '-')
+ if name not in HOOKS:
+ HOOKS[name] = []
+ HOOKS[name].append(func)
+ return func
+ return register
+
+def get_hooks(name, disabled=False):
+ name = name.replace('_', '-')
+ for pattern in DISABLED_HOOKS:
+ if fnmatch(name, pattern):
+ disabled = 1
+ for pattern in ENABLED_HOOKS:
+ if fnmatch(name, pattern):
+ disabled = 0
+ if disabled:
+ return []
+ return HOOKS.get(name, [])
+
+def call_hooks(name, *args, **kwargs):
+ name = name.replace('_', '-')
+ prev_hook = env.hook
+ env.hook = name
+ try:
+ for hook in get_hooks(name):
+ hook(*args, **kwargs)
+ finally:
+ env.hook = prev_hook
+
+hook.get = get_hooks
+hook.call = call_hooks
+hook.registry = HOOKS
+
+# ------------------------------------------------------------------------------
+# Timeout
+# ------------------------------------------------------------------------------
+
+class ProcessTimeout(object):
+ """Process timeout indicator."""
+
+ failed = 1
+ succeeded = 0
+
+ def __bool__(self):
+ return False
+
+ __nonzero__ = __bool__
+
+ def __str__(self):
+ return 'TIMEOUT'
+
+ __repr__ = __str__
+
+TIMEOUT = ProcessTimeout()
+
+class TimeoutException(Exception):
+ """An internal timeout exception raised on SIGALRM."""
+
+# ------------------------------------------------------------------------------
+# Proxy Boolean
+# ------------------------------------------------------------------------------
+
+class WarningBoolean(object):
+ """Proxy boolean to env.warning."""
+
+ def __bool__(self):
+ return env.warn_only
+
+ __nonzero__ = __bool__
+
+
+WarnOnly = WarningBoolean()
+
+# ------------------------------------------------------------------------------
+# Failure Handler
+# ------------------------------------------------------------------------------
+
+def handle_failure(cmd, warn_only):
+ if hasattr(cmd, '__name__'):
+ cmd = cmd.__name__ + '()'
+ message = 'Error running `%s`\n\n%s' % (cmd, indent(format_exc()))
+ if warn_only:
+ warn(message)
+ else:
+ abort(message)
+
+# ------------------------------------------------------------------------------
+# Shell Spec
+# ------------------------------------------------------------------------------
+
+class ShellSpec(object):
+ """Container class for shell spec variables."""
+
+ def __init__(self, **kwargs):
+ self.__dict__.update(kwargs)
+
+# ------------------------------------------------------------------------------
+# Response List
+# ------------------------------------------------------------------------------
+
+class ResponseList(list):
+ """Container class for response values."""
+
+ @classmethod
+ def new(klass, settings, value=None):
+ if value:
+ obj = klass(value)
+ else:
+ obj = klass()
+ obj._settings = settings
+ return obj
+
+ def ziphost(self):
+ for response, setting in zip(self, self._settings):
+ yield response, setting['host_string']
+
+ def zipsetting(self):
+ for response, setting in zip(self, self._settings):
+ yield response, setting
+
+ @property
+ def settings(self):
+ return self._settings[:]
+
+# ------------------------------------------------------------------------------
+# Execute Operation
+# ------------------------------------------------------------------------------
+
+DEFAULT_SCRIPT_NAME = 'fab.%s' % uuid4()
+
+def execute(
+ script, name=None, verbose=True, shell=True, pty=True, combine_stderr=True,
+ dir=None
+ ):
+ """Run arbitrary scripts on a remote host."""
+
+ script = dedent(script).strip()
+ if verbose:
+ prefix = "[%s]" % env.host_string
+ if env.colors:
+ prefix = env.color_settings['host_prefix'](prefix)
+ print("%s run: %s" % (prefix, name or script))
+ name = name or DEFAULT_SCRIPT_NAME
+ with hide('running', 'stdout', 'stderr'):
+ run('cat > ' + name + ' << FABEND\n' + script + '\nFABEND\n', dir=dir)
+ run('chmod +x ' + name, dir=dir)
+ try:
+ if verbose > 1:
+ with show('stdout', 'stderr'):
+ out = run('./' + name, shell, pty, combine_stderr, dir)
+ else:
+ out = run('./' + name, shell, pty, combine_stderr, dir)
+ finally:
+ run('rm ' + name, dir=dir)
+ return out
+
+# ------------------------------------------------------------------------------
+# Core Context Class
+# ------------------------------------------------------------------------------
+
+class ContextRunner(object):
+ """A convenience class to support operations on initialised contexts."""
+
+ def __init__(self, *args, **kwargs):
+ if kwargs and 'settings' in kwargs:
+ self.ctx = ('<sample>',)
+ self._settings = kwargs['settings']
+ if args:
+ if len(args) == 1 and not isinstance(args[0], basestring):
+ args = tuple(args[0])
+ self.ctx = args
+ self._settings = env.get_settings(args)
+ else:
+ if env.ctx:
+ self.ctx = env.ctx
+ self._settings = env.get_settings(env.ctx)
+ else:
+ self.ctx = ()
+ self._settings = []
+
+ def execute(
+ self, script, name=None, verbose=True, shell=True, pty=True,
+ combine_stderr=True, dir=None
+ ):
+ ctx, e = self.ctx, execute
+ settings_list = self._settings
+ responses = ResponseList.new(settings_list); out = responses.append
+ for kwargs in settings_list:
+ with settings(ctx=ctx, **kwargs):
+ out(e(script, name, verbose, shell, pty, combine_stderr, dir))
+ return responses
+
+ def local(self, command, capture=True, dir=None, format=True):
+ ctx, l = self.ctx, local
+ settings_list = self._settings
+ responses = ResponseList.new(settings_list); out = responses.append
+ for kwargs in settings_list:
+ with settings(ctx=ctx, **kwargs):
+ out(l(command, capture, dir, format))
+ return responses
+
+ def reboot(self, wait):
+ ctx, r = self.ctx, reboot
+ settings_list = self._settings
+ responses = ResponseList.new(settings_list); out = responses.append
+ for kwargs in settings_list:
+ with settings(ctx=ctx, **kwargs):
+ out(r(wait))
+ return responses
+
+ def run(
+ self, command, shell=True, pty=True, combine_stderr=True, dir=None,
+ format=True, warn_only=WarnOnly
+ ):
+ ctx = self.ctx
+ settings_list = self._settings
+ responses = ResponseList.new(settings_list); out = responses.append
+ if isinstance(command, basestring):
+ r = run
+ for kwargs in settings_list:
+ with settings(ctx=ctx, warn_only=warn_only, **kwargs):
+ out(r(command, shell, pty, combine_stderr, dir, format))
+ else:
+ for kwargs in settings_list:
+ with settings(ctx=ctx, warn_only=warn_only, **kwargs):
+ try:
+ out(command())
+ except Exception, error:
+ out(error)
+ handle_failure(command, warn_only)
+ return responses
+
+ def shell(
+ self, builtins=SHELL_BUILTINS, shell=True, pty=True,
+ combine_stderr=True, dir=None, format=True, warn_only=True
+ ):
+ ctx = self.ctx
+ settings_list = self._settings
+ if not settings_list:
+ return
+ global SHELL_HISTORY_FILE
+ if (not SHELL_HISTORY_FILE) and readline:
+ SHELL_HISTORY_FILE = expanduser(env.shell_history_file)
+ try:
+ readline.read_history_file(SHELL_HISTORY_FILE)
+ except IOError:
+ pass
+ atexit.register(readline.write_history_file, SHELL_HISTORY_FILE)
+ fastprint("shell mode\n\n", 'system')
+ spec = ShellSpec(
+ shell=shell, pty=pty, combine_stderr=combine_stderr, dir=dir,
+ format=format
+ )
+ r = run
+ count = 0
+ prefix = '>> '
+ if env.colors:
+ prefix = env.color_settings['prefix'](prefix)
+ try:
+ while 1:
+ try:
+ command = raw_input(prefix).strip()
+ except EOFError:
+ raise KeyboardInterrupt
+ if not command:
+ continue
+ builtin_cmd = 0
+ if command.startswith('.'):
+ if (len(command) > 1) and command[1].isalpha():
+ builtin_cmd = 1
+ if builtin_cmd:
+ command = command.split(' ', 1)
+ if len(command) == 1:
+ command = command[0]
+ arg = ''
+ else:
+ command, arg = command
+ command = command[1:].strip()
+ if not command:
+ continue
+ command = command.replace('_', '-')
+ if command not in builtins:
+ warn("Couldn't find builtin command %r" % command)
+ continue
+ command = builtins[command]
+ if hasattr(command, '__single__'):
+ with settings(ctx=ctx, warn_only=warn_only):
+ try:
+ command(spec, arg)
+ except Exception:
+ handle_failure(command, warn_only)
+ continue
+ for kwargs in settings_list:
+ with settings(ctx=ctx, warn_only=warn_only, **kwargs):
+ try:
+ if builtin_cmd:
+ try:
+ command(spec, arg)
+ except Exception:
+ handle_failure(command, warn_only)
+ else:
+ r(command, spec.shell, spec.pty,
+ spec.combine_stderr, spec.dir, spec.format)
+ except KeyboardInterrupt:
+ print
+ count += 1
+ if count > 2:
+ raise KeyboardInterrupt
+ count = 0
+ except KeyboardInterrupt:
+ print
+ print
+ fastprint("shell mode terminated\n", 'system')
+
+ def sudo(
+ self, command, shell=True, pty=True, combine_stderr=True, user=None,
+ dir=None, format=True
+ ):
+ ctx, s = self.ctx, sudo
+ settings_list = self._settings
+ responses = ResponseList.new(settings_list); out = responses.append
+ for kwargs in settings_list:
+ with settings(ctx=ctx, **kwargs):
+ out(s(command, shell, pty, combine_stderr, user, dir, format))
+ return responses
+
+ if forkable:
+
+ def multilocal(
+ self, command, capture=True, dir=None, format=True, warn_only=True,
+ condensed=False, quiet_exit=True, laggards_timeout=None,
+ wait_for=None
+ ):
+ def run_local():
+ return local(command, capture, dir, format)
+ return self.multirun(
+ run_local, warn_only=warn_only, condensed=condensed,
+ quiet_exit=quiet_exit, laggards_timeout=laggards_timeout,
+ wait_for=wait_for
+ )
+
+ def multisudo(
+ self, command, shell=True, pty=True, combine_stderr=True, user=None,
+ dir=None, format=True, warn_only=True, condensed=False,
+ quiet_exit=True, laggards_timeout=None, wait_for=None
+ ):
+ def run_sudo():
+ return sudo(
+ command, shell, pty, combine_stderr, user, dir, format
+ )
+ return self.multirun(
+ run_sudo, warn_only=warn_only, condensed=condensed,
+ quiet_exit=quiet_exit, laggards_timeout=laggards_timeout,
+ wait_for=wait_for
+ )
+
+ def multirun(
+ self, command, shell=True, pty=True, combine_stderr=True, dir=None,
+ format=True, warn_only=True, condensed=False, quiet_exit=True,
+ laggards_timeout=None, wait_for=None
+ ):
+ settings_list = self._settings
+ if not settings_list:
+ return ResponseList.new(settings_list)
+ if laggards_timeout:
+ if not isinstance(laggards_timeout, int):
+ raise ValueError(
+ "The laggards_timeout parameter must be an int."
+ )
+ if isinstance(wait_for, float):
+ if not 0.0 <= wait_for <= 1.0:
+ raise ValueError(
+ "A float wait_for needs to be between 0.0 and 1.0"
+ )
+ wait_for = int(ceil(wait_for * len(settings_list)))
+ env.disable_char_buffering = 1
+ try:
+ return self._multirun(
+ command, settings_list, shell, pty, combine_stderr, dir,
+ format, warn_only, condensed, quiet_exit, laggards_timeout,
+ wait_for
+ )
+ finally:
+ env.disable_char_buffering = 0
+
+ def _multirun(
+ self, command, settings_list, shell, pty, combine_stderr, dir,
+ format, warn_only, condensed, quiet_exit, laggards_timeout,
+ wait_for
+ ):
+
+ callable_command = hasattr(command, '__call__')
+ done = 0
+ idx = 0
+ ctx = self.ctx
+ processes = {}
+ total = len(settings_list)
+ pool_size = env.multirun_pool_size
+ socket_path = '/tmp/fab.%s' % uuid4()
+
+ server = socket(AF_UNIX, SOCK_STREAM)
+ server.bind(socket_path)
+ server.listen(pool_size)
+
+ for client_id in range(min(pool_size, total)):
+ from_parent, to_child = pipe()
+ pid = fork()
+ if pid:
+ processes[client_id] = [from_parent, to_child, pid, idx]
+ idx += 1
+ write(to_child, pack('H', idx))
+ else:
+ atfork()
+ def die(*args):
+ if quiet_exit:
+ output.status = False
+ sys.exit()
+ signal(SIGALRM, die)
+ if condensed:
+ sys.__ori_stdout__ = sys.stdout
+ sys.__ori_stderr__ = sys.stderr
+ sys.stdout = sys.stderr = DEVNULL
+ while 1:
+ alarm(env.multirun_child_timeout)
+ data = read(from_parent, INDEX_HEADER_SIZE)
+ alarm(0)
+ idx = unpack('H', data)[0] - 1
+ if idx == -1:
+ die()
+ try:
+ if callable_command:
+ with settings(
+ ctx=ctx, warn_only=warn_only,
+ **settings_list[idx]
+ ):
+ try:
+ response = command()
+ except Exception, error:
+ handle_failure(command, warn_only)
+ response = error
+ else:
+ with settings(
+ ctx=ctx, warn_only=warn_only,
+ **settings_list[idx]
+ ):
+ response = run(
+ command, shell, pty, combine_stderr,
+ dir, format
+ )
+ except BaseException, error:
+ response = error
+ client = socket(AF_UNIX, SOCK_STREAM)
+ client.connect(socket_path)
+ client.send(dumps((client_id, idx, response)))
+ client.close()
+
+
+ if laggards_timeout:
+ break_early = 0
+ responses = [TIMEOUT] * total
+ def timeout_handler(*args):
+ raise TimeoutException
+ original_alarm_handler = signal(SIGALRM, timeout_handler)
+ total_waited = 0.0
+ else:
+ responses = [None] * total
+
+ if condensed:
+ prefix = '[multirun]'
+ if env.colors:
+ prefix = env.color_settings['prefix'](prefix)
+ stdout = sys.stdout
+ if callable_command:
+ command = '%s()' % command.__name__
+ else:
+ command = command
+ if total < pool_size:
+ print (
+ "%s Running %r on %s hosts" % (prefix, command, total)
+ )
+ else:
+ print (
+ "%s Running %r on %s hosts with pool of %s" %
+ (prefix, command, total, pool_size)
+ )
+ template = "%s %%s/%s completed ..." % (prefix, total)
+ info = template % 0
+ written = len(info) + 1
+ stdout.write(info)
+ stdout.flush()
+
+ while done < total:
+ if laggards_timeout:
+ try:
+ if wait_for and done >= wait_for:
+ wait_start = time()
+ alarm(laggards_timeout)
+ conn, addr = server.accept()
+ except TimeoutException:
+ if not wait_for:
+ break_early= 1
+ break
+ if done >= wait_for:
+ break_early = 1
+ break
+ continue
+ else:
+ alarm(0)
+ if wait_for and done >= wait_for:
+ total_waited += time() - wait_start
+ if total_waited > laggards_timeout:
+ break_early = 1
+ break
+ else:
+ conn, addr = server.accept()
+ stream = []; buffer = stream.append
+ while 1:
+ try:
+ data = conn.recv(1024)
+ except socketerror, errmsg:
+ if errmsg.errno in [EAGAIN, EPIPE, EINTR]:
+ continue
+ raise
+ if not data:
+ break
+ buffer(data)
+ client_id, resp_idx, response = loads(''.join(stream))
+ responses[resp_idx] = response
+ done += 1
+ spec = processes[client_id]
+ if idx < total:
+ spec[3] = idx
+ idx += 1
+ write(spec[1], pack('H', idx))
+ else:
+ spec = processes.pop(client_id)
+ write(spec[1], pack('H', 0))
+ close(spec[0])
+ close(spec[1])
+ if condensed:
+ stdout.write('\x08' * written)
+ print (
+ "%s Finished on %s" %
+ (prefix, settings_list[resp_idx]['host_string'])
+ )
+ if done == total:
+ info = "%s %s/%s completed successfully!" % (
+ prefix, done, done
+ )
+ else:
+ info = template % done
+ written = len(info) + 1
+ stdout.write(info)
+ stdout.flush()
+
+ if laggards_timeout:
+ if break_early:
+ for spec in processes.itervalues():
+ kill(spec[2], SIGTERM)
+ if condensed:
+ stdout.write('\x08' * written)
+ info = "%s %s/%s completed ... laggards discarded!" % (
+ prefix, done, total
+ )
+ stdout.write(info)
+ stdout.flush()
+ signal(SIGALRM, original_alarm_handler)
+
+ if condensed:
+ stdout.write('\n')
+ stdout.flush()
+
+ server.close()
+ remove(socket_path)
+
+ return ResponseList.new(settings_list, responses)
+
+ else:
+
+ def multilocal(self, *args, **kwargs):
+ abort("multilocal is not supported on this setup")
+
+ def multirun(self, *args, **kwargs):
+ abort("multirun is not supported on this setup")
+
+ def multisudo(self, *args, **kwargs):
+ abort("multisudo is not supported on this setup")
+
+ def select(self, filter):
+ if isinstance(filter, int):
+ return ContextRunner(settings=sample(self._settings, filter))
+ return ContextRunner(settings=filter(self._settings[:]))
+
+ @property
+ def settings(self):
+ return self._settings[:]
+
+# ------------------------------------------------------------------------------
+# Utility API Functions
+# ------------------------------------------------------------------------------
+
+def failed(responses):
+ """Utility function that returns True if any of the responses failed."""
+
+ return any(isinstance(resp, Exception) or resp.failed for resp in responses)
+
+def succeeded(responses):
+ """Utility function that returns True if the responses all succeeded."""
+
+ return all(
+ (not isinstance(resp, Exception)) and resp.succeeded
+ for resp in responses
+ )
+
+def shell(name_or_func=None, single=False):
+ """Decorator to register shell builtin commands."""
+
+ if name_or_func:
+ if isinstance(name_or_func, basestring):
+ name = name_or_func
+ func = None
+ else:
+ name = name_or_func.__name__
+ func = name_or_func
+ else:
+ name = func = None
+ if func:
+ SHELL_BUILTINS[name.replace('_', '-')] = func
+ if single:
+ func.__single__ = 1
+ return func
+ def __decorate(func):
+ SHELL_BUILTINS[(name or func.__name__).replace('_', '-')] = func
+ if single:
+ func.__single__ = 1
+ return func
+ return __decorate
+
+# ------------------------------------------------------------------------------
+# Default Shell Builtins
+# ------------------------------------------------------------------------------
+
+@shell(single=True)
+def info(spec, arg):
+ """list the hosts and the current context"""
+
+ print
+ print "Context:"
+ print
+ print "\n".join(" %s" % ctx for ctx in env.ctx)
+ print
+ print "Hosts:"
+ print
+ for setting in env().settings:
+ print " ", setting['host_string']
+ print
+
+@shell(single=True)
+def cd(spec, arg):
+ """change to a new working directory"""
+
+ arg = arg.strip()
+ if arg:
+ if isabs(arg):
+ spec.dir = arg
+ elif arg.startswith('~'):
+ spec.dir = expanduser(arg)
+ else:
+ if spec.dir:
+ spec.dir = join(spec.dir, arg)
+ else:
+ spec.dir = join(getcwd(), arg)
+ spec.dir = normpath(spec.dir)
+ print "Switched to:", spec.dir
+ else:
+ spec.dir = None
+
+@shell('local', single=True)
+def builtin_local(spec, arg):
+ """run the command locally"""
+
+ local(arg, capture=0, dir=spec.dir, format=spec.format)
+
+@shell('sudo')
+def builtin_sudo(spec, arg):
+ """run the sudoed command on remote hosts"""
+
+ return sudo(
+ arg, spec.shell, spec.pty, spec.combine_stderr, None, spec.dir,
+ spec.format
+ )
+
+@shell(single=True)
+def toggle_format(spec, arg):
+ """toggle string formatting support"""
+
+ format = spec.format
+ if format:
+ spec.format = False
+ print "Formatting disabled."
+ else:
+ spec.format = True
+ print "Formatting enabled."
+
+@shell(single=True)
+def multilocal(spec, arg):
+ """run the command in parallel locally for each host"""
+
+ def run_local():
+ return local(arg, capture=0, dir=spec.dir, format=spec.format)
+
+ env().multirun(
+ run_local, spec.shell, spec.pty, spec.combine_stderr, spec.dir,
+ spec.format, quiet_exit=1
+ )
+
+@shell(single=True)
+def multirun(spec, arg):
+ """run the command in parallel on the various hosts"""
+
+ env().multirun(
+ arg, spec.shell, spec.pty, spec.combine_stderr, spec.dir, spec.format,
+ quiet_exit=1
+ )
+
+@shell(single=True)
+def multisudo(spec, arg):
+ """run the sudoed command in parallel on the various hosts"""
+
+ def run_sudo():
+ return sudo(
+ arg, spec.shell, spec.pty, spec.combine_stderr, None, spec.dir,
+ spec.format
+ )
+
+ env().multirun(
+ run_sudo, spec.shell, spec.pty, spec.combine_stderr, spec.dir,
+ spec.format, quiet_exit=1
+ )
+
+@shell(single=True)
+def help(spec, arg):
+ """display the list of available builtin commands"""
+
+ max_len = max(len(x) for x in SHELL_BUILTINS)
+ max_width = 80 - max_len - 5
+ print
+ print "Available Builtins:"
+ print
+ for builtin in sorted(SHELL_BUILTINS):
+ padding = (max_len - len(builtin)) * ' '
+ docstring = SHELL_BUILTINS[builtin].__doc__ or ''
+ if len(docstring) > max_width:
+ docstring = docstring[:max_width-3] + "..."
+ print " %s%s %s" % (padding, builtin, docstring)
+ print
+
+# ------------------------------------------------------------------------------
+# Monkey-Patch The Global Env Object
+# ------------------------------------------------------------------------------
+
+env.__dict__['_env_mgr'] = EnvManager
+env.__dict__['_ctx_class'] = ContextRunner
+env.get_settings = get_settings
+
+def __env_getattr__(self, key):
+ if key.isupper():
+ return self._env_mgr.for_var(key)
+ try:
+ return self[key]
+ except KeyError:
+ raise AttributeError(key)
+
+def __env_call__(self, *args, **kwargs):
+ return self._ctx_class(*args, **kwargs)
+
+env.__class__.__getattr__ = __env_getattr__
+env.__class__.__call__ = __env_call__
+
+# ------------------------------------------------------------------------------
+# Readline Completer
+# ------------------------------------------------------------------------------
+
+binaries_on_path = []
+
+def get_binaries_on_path():
+ env_path = environ.get('PATH')
+ if not env_path:
+ return
+ append = binaries_on_path.append
+ for path in env_path.split(pathsep):
+ path = path.strip()
+ if not path:
+ continue
+ if not isdir(path):
+ continue
+ for file in listdir(path):
+ file_path = join(path, file)
+ if access(file_path, X_OK):
+ append(file)
+ binaries_on_path.sort()
+
+def complete(text, state, matches=[], binaries={}):
+ if not state:
+ if text.startswith('.'):
+ text = text[1:]
+ matches[:] = [
+ '.' + builtin + ' '
+ for builtin in SHELL_BUILTINS if builtin.startswith(text)
+ ]
+ elif text.startswith('{'):
+ text = text[1:]
+ matches[:] = [
+ '{' + prop + '}'
+ for prop in env if prop.startswith(text)
+ ]
+ else:
+ if not binaries_on_path:
+ get_binaries_on_path()
+ matches[:] = []; append = matches.append
+ for file in binaries_on_path:
+ if file.startswith(text):
+ append(file)
+ else:
+ if matches:
+ break
+ try:
+ return matches[state]
+ except IndexError:
+ return
+
+try:
+ import readline
+except ImportError:
+ readline = None
+else:
+ readline.set_completer_delims(' \t\n')
+ readline.set_completer(complete)
+ readline.parse_and_bind('tab: complete')
+
+# ------------------------------------------------------------------------------
+# Task Decorator
+# ------------------------------------------------------------------------------
+
+def task(*args, **kwargs):
+ """Decorate a callable as being a task."""
+
+ display = kwargs.get('display', 1)
+ if args:
+ if hasattr(args[0], '__call__'):
+ func = args[0]
+ func.__task__ = 1
+ if not display:
+ func.__hide__ = 1
+ return func
+ ctx = args
+ if len(ctx) == 1 and not isinstance(ctx[0], basestring):
+ ctx = tuple(args[0])
+ else:
+ ctx = ()
+
+ def __task(__func):
+ __func.__ctx__ = ctx
+ __func.__task__ = 1
+ if not display:
+ __func.__hide__ = 1
+ return __func
+
+ return __task
+
+# ------------------------------------------------------------------------------
+# Stages Support
+# ------------------------------------------------------------------------------
+
+def set_env_stage_command(tasks, stage):
+ if stage in tasks:
+ return
+ def set_stage():
+ """Set the environment to %s.""" % stage
+ puts('env.stage = %s' % stage, 'system')
+ env.stage = stage
+ config_file = env.config_file
+ if config_file:
+ if not isinstance(config_file, basestring):
+ config_file = '%s.yaml'
+ try:
+ env.config_file = config_file % stage
+ except TypeError:
+ env.config_file = config_file
+ set_stage.__hide__ = 1
+ set_stage.__name__ = stage
+ set_stage.__task__ = 1
+ tasks[stage] = set_stage
+ return set_stage
+
+# ------------------------------------------------------------------------------
+# Global Defaults Initialisation
+# ------------------------------------------------------------------------------
+
+def setup_defaults(path=None):
+ """Initialise ``env`` and ``output`` to default values."""
+
+ env.update({
+ 'again_prompt': 'Sorry, try again.',
+ 'always_use_pty': True,
+ 'colors': False,
+ 'color_settings': {
+ 'abort': yellow,
+ 'error': yellow,
+ 'finish': cyan,
+ 'host_prefix': green,
+ 'prefix': red,
+ 'prompt': blue,
+ 'task': red,
+ 'warn': yellow
+ },
+ 'combine_stderr': True,
+ 'command': None,
+ 'command_prefixes': [],
+ 'config_file': None,
+ 'ctx': (),
+ 'cwd': '',
+ 'disable_known_hosts': False,
+ 'echo_stdin': True,
+ 'hook': None,
+ 'host': None,
+ 'host_string': None,
+ 'key_filename': None,
+ 'lcwd': '',
+ 'multirun_child_timeout': 10,
+ 'multirun_pool_size': 10,
+ 'no_agent': False,
+ 'no_keys': False,
+ 'output_prefix': True,
+ 'password': None,
+ 'passwords': {},
+ 'port': None,
+ 'reject_unknown_hosts': False,
+ 'shell': '/bin/bash -l -c',
+ 'shell_history_file': '~/.bolt-shell-history',
+ 'sudo_prefix': "sudo -S -p '%s' ",
+ 'sudo_prompt': 'sudo password:',
+ 'use_shell': True,
+ 'user': _get_system_username(),
+ 'warn_only': False
+ })
+
+ output.update({
+ 'aborts': True,
+ 'debug': False,
+ 'running': True,
+ 'status': True,
+ 'stderr': True,
+ 'stdout': True,
+ 'user': True,
+ 'warnings': True,
+ }, {
+ 'everything': ['output', 'running', 'user', 'warnings'],
+ 'output': [ 'stderr', 'stdout']
+ })
+
+ # Load defaults from a YAML file.
+ if path and exists(path):
+ fileobj = open(path, 'rb')
+ mapping = load_yaml(fileobj.read())
+ if not isinstance(mapping, dict):
+ abort(
+ "Got a %r value when loading %r. Mapping expected." %
+ (type(mapping), path)
+ )
+ env.update(mapping)
+
+# ------------------------------------------------------------------------------
+# Task Runner Initialiser
+# ------------------------------------------------------------------------------
+
+def init_task_runner(filename, cwd):
+ """Return a TaskRunner initialised from the located Boltfile."""
+
+ cwd = abspath(cwd)
+ if sep in filename:
+ path = join(cwd, filename)
+ if not exists(path):
+ abort("Couldn't find Boltfile: %s" % filename)
+ else:
+ prev = None
+ while cwd:
+ if cwd == prev:
+ abort("Couldn't find Boltfile: %s" % filename)
+ path = join(cwd, filename)
+ if exists(path):
+ break
+ prev = cwd
+ cwd = dirname(cwd)
+
+ directory = dirname(path)
+ if directory not in sys.path:
+ sys.path.insert(0, directory)
+
+ chdir(directory)
+
+ sys.dont_write_bytecode = 1
+ boltfile = load_source('boltfile', path)
+ sys.dont_write_bytecode = 0
+
+ tasks = dict(
+ (var.replace('_', '-'), obj) for var, obj in vars(boltfile).items()
+ if hasattr(obj, '__task__')
+ )
+
+ stages = environ.get('BOLT_STAGES', env.get('stages'))
+ if stages:
+ if isinstance(stages, basestring):
+ stages = [stage.strip() for stage in stages.split(',')]
+ env.stages = stages
+ for stage in stages:
+ set_env_stage_command(tasks, stage)
+
+ return TaskRunner(directory, path, boltfile.__doc__, tasks)
+
+# ------------------------------------------------------------------------------
+# Task Runner
+# ------------------------------------------------------------------------------
+
+class TaskRunner(object):
+ """Task runner encapsulation."""
+
+ def __init__(self, directory, path, docstring, tasks):
+ self.directory = directory
+ self.path = path
+ self.docstring = docstring
+ self.tasks = tasks
+
+ def display_listing(self):
+ """Print a listing of the available tasks."""
+
+ docstring = self.docstring
+ if docstring:
+ docstring = docstring.strip()
+ if docstring:
+ print(docstring + '\n')
+
+ print("Available tasks:\n")
+
+ tasks = self.tasks
+ width = max(map(len, tasks)) + 3
+
+ for name in sorted(tasks):
+ task = tasks[name]
+ if hasattr(task, '__hide__'):
+ continue
+ padding = " " * (width - len(name))
+ print(" %s%s%s" % (name, padding, (task.__doc__ or "")[:80]))
+ print
+
+ if 'stages' in env:
+ print 'Available environments:'
+ print
+ for stage in env.stages:
+ print ' %s' % stage
+ print
+
+ call_hooks('listing.display')
+
+ def execute_task(self, name, args, kwargs, ctx):
+ """Execute the given task."""
+
+ task = self.tasks[name]
+ env.command = name
+ if output.running:
+ msg = "running task: %s" % name
+ prefix = '[system] '
+ if env.colors:
+ prefix = env.color_settings['prefix'](prefix)
+ print(prefix + msg)
+
+ if not ctx:
+ ctx = getattr(task, '__ctx__', None)
+
+ if ctx:
+ with settings(ctx=ctx):
+ task(*args, **kwargs)
+ return
+
+ task(*args, **kwargs)
+
+ def run(self, spec):
+ """Execute the various tasks given in the spec list."""
+
+ try:
+
+ if output.debug:
+ names = ", ".join(info[0] for info in spec)
+ print("Tasks to run: %s" % names)
+
+ call_hooks('commands.before', self.tasks, spec)
+
+ # Initialise the default stage if none are given as the first task.
+ if 'stages' in env:
+ if spec[0][0] not in env.stages:
+ self.execute_task(env.stages[0], (), {}, None)
+ else:
+ self.execute_task(*spec.pop(0))
+
+ # Load the config YAML file if specified.
+ if env.config_file:
+ config_path = realpath(expanduser(env.config_file))
+ config_path = join(self.directory, config_path)
+ config_file = open(config_path, 'rb')
+ config = load_yaml(config_file.read())
+ if not config:
+ env.config = AttrDict()
+ elif not isinstance(config, dict):
+ abort("Invalid config file found at %s" % config_path)
+ else:
+ env.config = AttrDict(config)
+ config_file.close()
+
+ call_hooks('config.loaded')
+
+ # Execute the tasks in order.
+ for info in spec:
+ self.execute_task(*info)
+
+ if output.status:
+ msg = "\nDone."
+ if env.colors:
+ msg = env.color_settings['finish'](msg)
+ print(msg)
+
+ except SystemExit:
+ raise
+ except KeyboardInterrupt:
+ if output.status:
+ msg = "\nStopped."
+ if env.colors:
+ msg = env.color_settings['finish'](msg)
+ print >> sys.stderr, msg
+ sys.exit(1)
+ except:
+ sys.excepthook(*sys.exc_info())
+ sys.exit(1)
+ finally:
+ call_hooks('commands.after')
+ disconnect_all()
+
+# ------------------------------------------------------------------------------
+# Script Runner
+# ------------------------------------------------------------------------------
+
+def main(argv=None):
+ """Handle the bolt command line call."""
+
+ if argv is None:
+ argv = sys.argv[1:]
+
+ op = OptionParser(
+ usage="bolt <command-1> <command-2> ... [options]",
+ )
+
+ op.add_option(
+ '-v', '--version', action='store_true', default=False,
+ help="show program's version number and exit"
+ )
+
+ op.add_option(
+ '-f', dest='file', default="Boltfile",
+ help="set the name or path of the bolt file [Boltfile]"
+ )
+
+ op.add_option(
+ '-d', dest='defaults_file', default=_rc_path(),
+ help="set the path of the defaults file [~/.bolt.yaml]"
+ )
+
+ op.add_option(
+ '-i', dest='identity', action='append', default=None,
+ help="path to SSH private key file(s) -- may be repeated"
+ )
+
+ op.add_option(
+ '--hide', metavar='LEVELS',
+ help="comma-separated list of output levels to hide"
+ )
+
+ op.add_option(
+ '--show', metavar='LEVELS',
+ help="comma-separated list of output levels to show"
+ )
+
+ op.add_option(
+ '--disable', metavar='HOOKS',
+ help="comma-separated list of hooks to disable"
+ )
+
+ op.add_option(
+ '--enable', metavar='HOOKS',
+ help="comma-separated list of hooks to enable"
+ )
+
+ op.add_option(
+ '--list', action='store_true', default=False,
+ help="show the list of available tasks and exit"
+ )
+
+ op.add_option(
+ '--no-pty', action='store_true', default=False,
+ help="do not use pseudo-terminal in run/sudo"
+ )
+
+ options, args = op.parse_args(argv)
+ setup_defaults(options.defaults_file)
+
+ # Load the Boltfile.
+ runner = init_task_runner(options.file, getcwd())
+
+ # Autocompletion support.
+ autocomplete_items = runner.tasks.keys()
+ if 'autocomplete' in env:
+ autocomplete_items += env.autocomplete
+
+ autocomplete(op, ListCompleter(autocomplete_items))
+
+ if options.version:
+ print("bolt %s" % __version__)
+ sys.exit()
+
+ if options.no_pty:
+ env.always_use_pty = False
+
+ if options.identity:
+ env.key_filename = options.identity
+
+ split_string = lambda s: filter(None, map(str.strip, s.split(',')))
+
+ # Handle output levels.
+ if options.show:
+ for level in split_string(options.show):
+ output[level] = True
+
+ if options.hide:
+ for level in split_string(options.hide):
+ output[level] = False
+
+ if output.debug:
+ print("Using Boltfile: %s" % runner.path)
+
+ # Handle hooks related options.
+ if options.disable:
+ for hook in split_string(options.disable):
+ DISABLED_HOOKS.append(hook)
+
+ if options.enable:
+ for hook in split_string(options.enable):
+ ENABLED_HOOKS.append(hook)
+
+ if options.list:
+ print('\n'.join(sorted(runner.tasks)))
+ sys.exit()
+
+ tasks = []
+ idx = 0
+
+ # Parse command line arguments.
+ for task in args:
+
+ # Initialise variables.
+ _args = []
+ _kwargs = {}
+ _ctx = None
+
+ # Handle +env flags.
+ if task.startswith('+'):
+ if ':' in task:
+ name, value = task[1:].split(':', 1)
+ env[name] = value
+ else:
+ env[task[1:]] = True
+ continue
+
+ # Handle @context specifiers.
+ if task.startswith('@'):
+ if not idx:
+ continue
+ ctx = (task[1:],)
+ existing = tasks[idx-1][3]
+ if existing:
+ new = list(existing)
+ new.extend(ctx)
+ ctx = tuple(new)
+ tasks[idx-1][3] = ctx
+ continue
+
+ # Handle tasks with parameters.
+ if ':' in task:
+ task, argstr = task.split(':', 1)
+ for pair in _escape_split(',', argstr):
+ k, _, v = pair.partition('=')
+ if _:
+ _kwargs[k] = v
+ else:
+ _args.append(k)
+
+ idx += 1
+ task_name = task.replace('_', '-')
+
+ if task_name not in runner.tasks:
+ abort("Task not found:\n\n%s" % indent(task))
+
+ tasks.append([task_name, _args, _kwargs, _ctx])
+
+ if not tasks:
+ runner.display_listing()
+ sys.exit()
+
+ runner.run(tasks)
+
+# ------------------------------------------------------------------------------
+# Self Runner
+# ------------------------------------------------------------------------------
+
+if __name__ == '__main__':
+ main()
1,516 bolt/fabcode.py
@@ -0,0 +1,1516 @@
+# Changes to this file by The Bolt Authors are in the Public Domain.
+# See the Bolt UNLICENSE file for details.
+
+# Copyright (c) 2009-2011, Christian Vest Hansen and Jeffrey E. Forcier
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+#
+# * Redistributions of source code must retain the above copyright notice,
+# this list of conditions and the following disclaimer.
+#
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""Support code derived from the Fabric codebase."""
+
+import sys
+
+from contextlib import closing, contextmanager, nested
+from fnmatch import filter as fnfilter
+from functools import wraps
+from getpass import getpass
+from glob import glob
+from hashlib import sha1
+from os import devnull, fdopen, getcwd, getuid, makedirs, remove, stat, walk
+from os.path import abspath, basename, dirname, exists, expanduser, isabs, isdir
+from os.path import join, split
+from re import compile as compile_regex, findall
+from select import select
+from socket import error as socketerror, gaierror, timeout
+from stat import S_ISDIR, S_ISLNK
+from subprocess import PIPE, Popen
+from tempfile import mkstemp
+from textwrap import dedent
+from threading import Thread
+from time import sleep
+from traceback import format_exc
+
+# ------------------------------------------------------------------------------
+# Platform Specific Imports
+# ------------------------------------------------------------------------------
+
+win32 = (sys.platform == 'win32')
+
+if win32:
+ import msvcrt
+else:
+ import fcntl
+ import struct
+ import termios
+ import tty
+
+# ------------------------------------------------------------------------------
+# Some Constants
+# ------------------------------------------------------------------------------
+
+IO_SLEEP = 0.01
+
+# ------------------------------------------------------------------------------
+# Container Classes
+# ------------------------------------------------------------------------------
+
+class AttrDict(dict):
+ """Dict subclass that allows for attribute access of key/values."""
+
+ def __getattr__(self, key):
+ try:
+ return self[key]
+ except KeyError:
+ raise AttributeError(key)
+
+ def __setattr__(self, key, value):
+ self[key] = value
+
+class AliasDict(AttrDict):
+ """Subclass that allows for "aliasing" of keys to other keys."""
+
+ def __init__(self, mapping=None, aliases={}):
+ self.update(mapping, aliases)
+
+ def __setitem__(self, key, value):
+ if key in self.aliases:
+ for aliased in self.aliases[key]:
+ self[aliased] = value
+ else:
+ return dict.__setitem__(self, key, value)
+
+ def expand_aliases(self, keys):
+ ret = []
+ for key in keys:
+ if key in self.aliases:
+ ret.extend(self.expand_aliases(self.aliases[key]))
+ else:
+ ret.append(key)
+ return ret
+
+ def update(self, mapping=None, aliases={}):
+ if mapping is not None:
+ for key in mapping:
+ dict.__setitem__(self, key, mapping[key])
+ dict.__setattr__(self, 'aliases', aliases)
+
+class EnvDict(AttrDict):
+ """Environment dictionary object."""
+
+# ------------------------------------------------------------------------------
+# Global Objects
+# ------------------------------------------------------------------------------
+
+env = EnvDict()
+output = AliasDict()
+
+# ------------------------------------------------------------------------------
+# Utility Functions
+# ------------------------------------------------------------------------------
+
+def stringify_env_var(var):
+ """Format the complete environment $VARIABLE setting string."""
+
+ key = result = '$%s' % var
+ for value, behaviour, sep in env.get(key, []):
+ if behaviour == 'append':
+ result = result + sep + '"' + value + '"'
+ elif behaviour == 'prepend':
+ result = '"' + value + '"' + sep + result
+ else:
+ result = '"' + value + '"'
+ return "%s=%s" % (var, result)
+
+def _get_system_username():
+ """Return the current system user."""
+
+ if not win32:
+ import pwd
+ return pwd.getpwuid(getuid())[0]
+ else:
+ import win32api
+ import win32security
+ import win32profile
+ return win32api.GetUserName()
+
+def _rc_path(rc_file='.bolt.yaml'):
+ """Return the platform-specific path for $HOME/.bolt.yaml."""
+
+ if not win32:
+ return expanduser("~/" + rc_file)
+ else:
+ from win32com.shell.shell import SHGetSpecialFolderPath
+ from win32com.shell.shellcon import CSIDL_PROFILE
+ return "%s/%s" % (SHGetSpecialFolderPath(0,CSIDL_PROFILE), rc_file)
+
+def _escape_split(sep, argstr):
+ """Split string, allowing for escaping of the separator."""
+
+ escaped_sep = r'\%s' % sep
+ if escaped_sep not in argstr:
+ return argstr.split(sep)
+
+ before, _, after = argstr.partition(escaped_sep)
+ startlist = before.split(sep)
+ unfinished = startlist[-1]
+ startlist = startlist[:-1]
+ endlist = _escape_split(sep, after)
+ unfinished += sep + endlist[0]
+ return startlist + [unfinished] + endlist[1:]
+
+# ------------------------------------------------------------------------------
+# Authentication Support
+# ------------------------------------------------------------------------------
+
+def get_password():
+ return env.passwords.get(env.host_string, env.password)
+
+def set_password(password):
+ env.password = env.passwords[env.host_string] = password
+
+# ------------------------------------------------------------------------------
+# Colours Support
+# ------------------------------------------------------------------------------
+
+def _bold_wrap_with(code):
+ def inner(text):
+ return "\033[1;%sm%s\033[0m" % (code, text)
+ return inner
+
+def _wrap_with(code):
+ def inner(text, bold=False):
+ c = code
+ if bold:
+ c = "1;%s" % c
+ return "\033[%sm%s\033[0m" % (c, text)
+ return inner
+
+bold_blue = _bold_wrap_with('34')
+bold_cyan = _bold_wrap_with('36')
+bold_green = _bold_wrap_with('32')
+bold_magenta = _bold_wrap_with('35')
+bold_red = _bold_wrap_with('31')
+bold_white = _bold_wrap_with('37')
+bold_yellow = _bold_wrap_with('33')
+
+blue = _wrap_with('34')
+cyan = _wrap_with('36')
+green = _wrap_with('32')
+magenta = _wrap_with('35')
+red = _wrap_with('31')
+white = _wrap_with('37')
+yellow = _wrap_with('33')
+
+# ------------------------------------------------------------------------------
+# Thread Handler
+# ------------------------------------------------------------------------------
+
+class ThreadHandler(object):
+ """Wrapper around worker threads."""
+
+ def __init__(self, name, callable, *args, **kwargs):
+ self.exception = None
+ def wrapper(*args, **kwargs):
+ try:
+ callable(*args, **kwargs)
+ except BaseException:
+ self.exception = sys.exc_info()
+ thread = Thread(None, wrapper, name, args, kwargs)
+ thread.setDaemon(True)
+ thread.start()
+ self.thread = thread
+
+# ------------------------------------------------------------------------------
+# Context Managers
+# ------------------------------------------------------------------------------
+
+@contextmanager
+def char_buffered(pipe):
+ """Force the local terminal ``pipe`` to be character, not line, buffered."""
+
+ if win32 or env.get('disable_char_buffering', 0) or not sys.stdin.isatty():
+ yield
+ else:
+ old_settings = termios.tcgetattr(pipe)
+ tty.setcbreak(pipe)
+ try:
+ yield
+ finally:
+ termios.tcsetattr(pipe, termios.TCSADRAIN, old_settings)
+
+def _set_output(groups, which):
+ previous = {}
+ for group in output.expand_aliases(groups):
+ previous[group] = output[group]
+ output[group] = which
+ yield
+ output.update(previous)
+
+@contextmanager
+def hide(*groups):
+ """Hide output from the given ``groups``."""
+
+ return _set_output(groups, False)
+
+@contextmanager
+def show(*groups):
+ """Show output from the given ``groups``."""
+
+ return _set_output(groups, True)
+
+@contextmanager
+def _setenv(**kwargs):
+ previous = {}
+ for key, value in kwargs.iteritems():
+ if key in env:
+ previous[key] = env[key]
+ env[key] = value
+ try:
+ yield
+ finally:
+ env.update(previous)
+
+def prefix(command):
+ """Prefix ``run``/``sudo`` calls with the given ``command`` plus ``&&``."""
+
+ return _setenv(command_prefixes=env.command_prefixes + [command])
+
+def settings(*ctxmanagers, **env_values):
+ """Nest the ``ctxmanagers`` and temporarily override the ``env_values``."""
+
+ managers = list(ctxmanagers)
+ if env_values:
+ managers.append(_setenv(**env_values))
+ return nested(*managers)
+
+def _change_cwd(which, path):
+ path = path.replace(' ', '\ ')
+ if env.get(which) and not path.startswith('/'):
+ new_cwd = env.get(which) + '/' + path
+ else:
+ new_cwd = path
+ return _setenv(**{which: new_cwd})
+
+def cd(path):
+ """Prefix run/sudo/get/put calls to run in the given remote ``path``."""
+
+ return _change_cwd('cwd', path)
+
+def lcd(path):
+ """Prefix local/get/put calls to run in the given local ``path``."""
+
+ return _change_cwd('lcwd', path)
+
+# ------------------------------------------------------------------------------
+# Formatters
+# ------------------------------------------------------------------------------
+
+def indent(text, spaces=4, strip=False):
+ """Return the ``text`` indented by the given number of ``spaces``."""
+
+ if not hasattr(text, 'splitlines'):
+ text = '\n'.join(text)
+ if strip:
+ text = dedent(text)
+ prefix = ' ' * spaces
+ output = '\n'.join(prefix + line for line in text.splitlines())
+ output = output.strip()
+ output = prefix + output
+ return output
+
+# ------------------------------------------------------------------------------
+# Modified Output Utilities
+# ------------------------------------------------------------------------------
+
+def abort(msg):
+ """Print the given ``msg`` and exit with status code 1."""
+
+ if output.aborts:
+ if env.colors:
+ abort_color = env.color_settings['abort']
+ print >> sys.stderr, abort_color("\nFatal error: " + str(msg))
+ print >> sys.stderr, abort_color("\nAborting.")
+ else:
+ print >> sys.stderr, "\nFatal error: " + str(msg)
+ print >> sys.stderr, "\nAborting."
+
+ sys.exit(1)
+
+def warn(msg):
+ """Print the given warning ``msg``."""
+
+ if output.warnings:
+ msg = "\nWarning: %s\n" % msg
+ if env.colors:
+ print >> sys.stderr, env.color_settings['warn'](msg)
+ else:
+ print >> sys.stderr, msg
+
+def puts(
+ text, prefix=None, end="\n", flush=False, show_host=True, format=True
+ ):
+ """Print the given ``text`` within the constraints of the output state."""
+
+ if output.user:
+ if prefix:
+ prefix = '[%s] ' % prefix
+ else:
+ prefix = ''
+ if show_host and env.host_string:
+ host_prefix = "[%s] " % env.host_string
+ else:
+ host_prefix = ''
+ if env.colors:
+ if prefix:
+ prefix = env.color_settings['prefix'](prefix)
+ if host_prefix:
+ host_prefix = env.color_settings['host_prefix'](host_prefix)
+ text = host_prefix + prefix + str(text) + end
+ if format:
+ text = text.format(**env)
+ sys.stdout.write(text)
+ if flush:
+ sys.stdout.flush()
+
+def fastprint(
+ text, prefix=False, end="", flush=True, show_host=False, format=True
+ ):
+ """Like ``puts``, but defaults to printing immediately."""
+
+ return puts(text, prefix, end, flush, show_host, format)
+
+# ------------------------------------------------------------------------------
+# Networking Support
+# ------------------------------------------------------------------------------
+
+try:
+ import warnings
+ warnings.simplefilter('ignore', DeprecationWarning)
+ import paramiko as ssh
+except ImportError:
+ abort(
+ """paramiko is a required module. Please install it:
+ $ sudo easy_install paramiko
+ """)
+
+HOST_PATTERN = r'((?P<user>.+)@)?(?P<host>[^:]+)(:(?P<port>\d+))?'
+HOST_REGEX = compile_regex(HOST_PATTERN)
+
+class HostConnectionCache(dict):
+ """Dict subclass allowing for caching of host connections/clients."""
+
+ def __getitem__(self, key):
+ user, host, port = normalize(key)
+ real_key = join_host_strings(user, host, port)
+ if real_key not in self:
+ self[real_key] = connect(user, host, port)
+ return dict.__getitem__(self, real_key)
+
+ def __delitem__(self, key):
+ return dict.__delitem__(self, join_host_strings(*normalize(key)))
+
+CONNECTIONS = HostConnectionCache()
+
+def default_channel():
+ """Return a channel object based on ``env.host_string``."""
+
+ return CONNECTIONS[env.host_string].get_transport().open_session()
+
+def normalize(host_string, omit_port=False):
+ """Normalize a ``host_string``, returning the explicit host, user, port."""
+
+ if not host_string:
+ return ('', '') if omit_port else ('', '', '')
+
+ r = HOST_REGEX.match(host_string).groupdict()
+ user = r['user'] or env.get('user')
+ host = r['host']
+ port = r['port'] or '22'
+ if omit_port:
+ return user, host
+
+ return user, host, port
+
+def denormalize(host_string):
+ """Strip default values for the given ``host_string``."""
+
+ r = HOST_REGEX.match(host_string).groupdict()
+ user = ''
+ if r['user'] is not None and r['user'] != env.user:
+ user = r['user'] + '@'
+ port = ''
+ if r['port'] is not None and r['port'] != '22':
+ port = ':' + r['port']
+ return user + r['host'] + port
+
+def join_host_strings(user, host, port=None):
+ """Turn user/host/port strings into ``user@host:port`` combined string."""
+
+ port_string = ''
+ if port:
+ port_string = ":%s" % port
+ return "%s@%s%s" % (user, host, port_string)
+
+def connect(user, host, port):
+ """Create and return a new SSHClient connected to the given host."""
+
+ client = ssh.SSHClient()
+
+ if not env.disable_known_hosts:
+ client.load_system_host_keys()
+
+ if not env.reject_unknown_hosts:
+ client.set_missing_host_key_policy(ssh.AutoAddPolicy())
+
+ connected = False
+ password = get_password()