Merge pull request #1 from aweber/publicize

Publicize

.gitignore

@@ -10,4 +10,3 @@ dist/
 .coverage
 *.sublime-project
 *.sublime-workspace
-LOCAL-VERSION

LICENSE

@@ -0,0 +1,25 @@
+Copyright (c) 2016 AWeber Communications
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+
+ * Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+ * Neither the name of Bandoleers nor the names of its contributors may be used
+   to endorse or promote products derived from this software without specific
+   prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
+IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
+BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
+OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

MANIFEST.in

@@ -1,3 +1,2 @@
-include LOCAL-VERSION
 graft docs
 graft requires

README.rst

@@ -4,12 +4,21 @@ Bandoleers
 This package contains useful command line tools that make deployment and
 local data bootstrapping less painful.
 
+:prep-it:
+    This simple utility iterates over data files in a sub-tree and loads
+    them into various backends such as Cassandra, RabbitMQ, and PostgreSQL.
+
+:wait-for:
+    This simple utility pings a URL periodically until it responds
+    successfully.  It supports HTTP, Cassandra, and Postgres URLs out of
+    the box.
+
 
 Quickstart Development Guide
 ----------------------------
 
 .. code:: bash
 
-    $ virtualenv
+    $ python3.4 -mvenv env
     $ . ./env/bin/activate
-    (env) $ pip install -qr requires/development.txt -i https://pypi.python.org/simple
+    (env) $ pip install -qr requires/development.txt

bandoleers/__init__.py

@@ -1,2 +1,2 @@
-version_info = (0, 3, 5)
+version_info = (0, 4, 0)
 __version__ = '.'.join(str(s) for s in version_info)

docs/conf.py

@@ -28,4 +28,5 @@
     'python': ('http://docs.python.org/3/', None),
     'rejected': ('http://rejected.readthedocs.org/en/latest/', None),
     'consulate': ('http://consulate.readthedocs.org/en/latest/', None),
+    'datastax': ('http://datastax.github.io/python-driver', None),
 }

docs/contributing.rst

@@ -11,23 +11,20 @@ run the test suite, update the documentation, and everything else that is
 involved in contributing.  The easiest way to do that is to create a virtual
 environment for your endevours::
 
-   $ pyvenv env
+   $ python3.4 -mvenv env
 
 Don't worry about writing code against previous versions of Python unless
-you you don't have a choice.  That is why we run our tests through `tox`_.
-If you don't have a choice, then install `virtualenv`_ to create the
-environment instead.  The next step is to install the development tools
-that this project uses.  These are listed in *requires/development.txt*::
+you you don't have a choice.  If you don't have a choice, then install
+`virtualenv`_ to create the environment instead.  The next step is to
+install the development tools that this project uses.  These are listed in
+*requires/development.txt*::
 
    $ env/bin/pip install -qr requires/development.txt
 
 At this point, you will have everything that you need to develop at your
 disposal.  *setup.py* is the swiss-army knife in your development tool
 chest.  It provides the following commands:
 
-**./setup.py nosetests**
-   Run the test suite using `nose`_ and generate a nice coverage report.
-
 **./setup.py build_sphinx**
    Generate the documentation using `sphinx`_.
 
@@ -37,81 +34,16 @@ chest.  It provides the following commands:
 If any of the preceding commands give you problems, then you will have to
 fix them **before** your pull request will be accepted.
 
-Running Tests
--------------
-The easiest (and quickest) way to run the test suite is to use the
-*nosetests* command.  It will run the test suite against the currently
-installed python version and report not only the test result but the
-test coverage as well::
-
-   $ ./setup.py nosetests
-
-   running nosetests
-   running egg_info
-   writing dependency_links to spoolconsumer.egg-info/dependency_links.txt
-   writing top-level names to spoolconsumer.egg-info/top_level.txt
-   writing spoolconsumer.egg-info/PKG-INFO
-   reading manifest file 'spoolconsumer.egg-info/SOURCES.txt'
-   reading manifest template 'MANIFEST.in'
-   warning: no previously-included files matching '__pycache__'...
-   warning: no previously-included files matching '*.swp' found ...
-   writing manifest file 'spoolconsumer.egg-info/SOURCES.txt'
-   ...
-
-   Name                       Stmts   Miss Branch BrMiss  Cover   Missing
-   ----------------------------------------------------------------------
-   ...
-   ----------------------------------------------------------------------
-   TOTAL                         95      2     59      2    97%
-   ----------------------------------------------------------------------
-   Ran 44 tests in 0.054s
-
-   OK
-
-That's the quick way to run tests.  The slightly longer way is to run
-the `tox`_ utility.  It will run the test suite against all of the
-supported python versions.  This is essentially what Travis-CI will do
-when you issue a pull request anyway::
-
-   $ env/bin/tox
-   py27 recreate: /.../spoolconsumer/build/tox/py27
-   GLOB sdist-make: /.../spoolconsumer/setup.py
-   py33 recreate: /.../spoolconsumer/build/tox/py33
-   py34 recreate: /.../spoolconsumer/build/tox/py34
-   py27 installdeps: -rtest-requirements.txt, mock
-   py33 installdeps: -rtest-requirements.txt
-   py34 installdeps: -rtest-requirements.txt
-   py27 inst: /.../spoolconsumer/build/tox/dist/spoolconsumer-0.0.0.zip
-   py27 runtests: PYTHONHASHSEED='2156646470'
-   py27 runtests: commands[0] | /../spoolconsumer/build/tox/py27/bin/nosetests
-   py33 inst: /../spoolconsumer/.build/tox/dist/spoolconsumer-0.0.0.zip
-   py34 inst: /../spoolconsumer/.build/tox/dist/spoolconsumer-0.0.0.zip
-   py33 runtests: PYTHONHASHSEED='2156646470'
-   py33 runtests: commands[0] | /.../spoolconsumer/build/tox/py33/bin/nosetests
-   py34 runtests: PYTHONHASHSEED='2156646470'
-   py34 runtests: commands[0] | /.../spoolconsumer/build/tox/py34/bin/nosetests
-   _________________________________ summary _________________________________
-     py27: commands succeeded
-     py33: commands succeeded
-     py34: commands succeeded
-     congratulations :)
-
-This is what you want to see.  Now you can make your modifications and keep
-the tests passing.
-
 Submitting a Pull Request
 -------------------------
-Once you have made your modifications, gotten all of the tests to pass,
-and added any necessary documentation, it is time to contribute back for
-posterity.  You've probably already cloned this repository and created a
-new branch.  If you haven't, then checkout what you have as a branch and
-roll back *master* to where you found it.  Then push your repository up
-to github and issue a pull request.  Describe your changes in the request,
-if Travis isn't too annoyed someone will review it, and eventually merge
-it back.
+Once you have made your modifications and added any necessary documentation,
+it is time to contribute back for posterity.  You've probably already cloned
+this repository and created a new branch.  If you haven't, then checkout what
+you have as a branch and roll back *master* to where you found it.  Then push
+your repository up to github and issue a pull request.  Describe your changes
+in the request and someone will review it, and eventually merge it and release
+a new version.
 
 .. _flake8: http://flake8.readthedocs.org/
-.. _nose: http://nose.readthedocs.org/
 .. _sphinx: http://sphinx-doc.org/
-.. _tox: http://testrun.org/tox/
 .. _virtualenv: http://virtualenv.pypa.io/

docs/environment.rst

@@ -0,0 +1,44 @@
+Environment Variables
+=====================
+
+.. envvar:: CASSANDRA_URI
+
+   Identifies a Cassandra cluster to connect to.  The general form is::
+
+      cassandra://HOST:PORT?property=value&property=value
+
+   The ``HOST`` portion is the DNS name or IP address of the Cassandra
+   server.  It is used as the ``contact_points`` parameter in the
+   :class:`cassandra.cluster.Cluster` initializer.  If it resolves to 
+   multiple addresses, then the entire list is passed to the initializer.
+   If the ``PORT`` is omitted, then it defaults to 9042.  The optional
+   "query string" contains properties that are passed to the initializer
+   as keyword parameters.
+
+.. envvar:: CONSUL_HOST
+
+   Identifies the IP address or DNS name of the Consul server.
+
+.. envvar:: CONSUL_PORT
+
+   Identifies the TCP port number that the Consul server is listening on.
+
+.. envvar:: RABBITMQ
+
+   The network location portion for the RabbitMQ HTTP API.  This is
+   inserted directly into a HTTP URL.
+
+.. envvar:: REDIS_URI
+
+   Identifies the redis server, port, and database to connect to.  This
+   value follows the IANA-registered `redis url`_ format.
+
+
+.. envvar:: PGSQL
+
+   Identifies the PostgreSQL server to connect to using the standard
+   `postgresql:// scheme`_
+
+
+.. _postgresql:// scheme: http://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-CONNSTRING
+.. _redis url: https://www.iana.org/assignments/uri-schemes/prov/redis

docs/history.rst

@@ -3,6 +3,10 @@
 Release History
 ===============
 
+`Next Release`_
+---------------
+- Infect the world!
+
 `0.3.5`_ (2016-01-26)
 ---------------------
 - Query parameters are now passed from cassandra:// URLs into the cluster
@@ -33,11 +37,11 @@ Release History
 - Initial release of the PrepIt package
 - Import @briank's work on prepit.
 
-
-.. _`0.1.0`: https://github.aweber.io/Platform/bandoleers/compare/0.0.0...0.1.0
-.. _`0.2.0`: https://github.aweber.io/Platform/bandoleers/compare/0.1.0...0.2.0
-.. _`0.2.1`: https://github.aweber.io/Platform/bandoleers/compare/0.2.0...0.2.1
-.. _`0.3.0`: https://github.aweber.io/Platform/bandoleers/compare/0.2.1...0.3.0
-.. _`0.3.3`: https://github.aweber.io/Platform/bandoleers/compare/0.3.0...0.3.3
-.. _`0.3.4`: https://github.aweber.io/Platform/bandoleers/compare/0.3.3...0.3.4
-.. _`0.3.5`: https://github.aweber.io/Platform/bandoleers/compare/0.3.4...0.3.5
+.. _Next Release: https://github.com/aweber/bandoleers/compare/0.3.5...HEAD
+.. _0.3.5: https://github.com/aweber/bandoleers/compare/0.3.4...0.3.5
+.. _0.3.4: https://github.com/aweber/bandoleers/compare/0.3.3...0.3.4
+.. _0.3.3: https://github.com/aweber/bandoleers/compare/0.3.0...0.3.3
+.. _0.3.0: https://github.com/aweber/bandoleers/compare/0.2.1...0.3.0
+.. _0.2.1: https://github.com/aweber/bandoleers/compare/0.2.0...0.2.1
+.. _0.2.0: https://github.com/aweber/bandoleers/compare/0.1.0...0.2.0
+.. _0.1.0: https://github.com/aweber/bandoleers/compare/0.0.0...0.1.0

docs/index.rst

@@ -6,12 +6,8 @@ Documentation
 .. toctree::
    :maxdepth: 2
 
+   prep-it
+   wait-for
+   environment
    history
    contributing
-
-Indices and tables
-==================
-
-* :ref:`genindex`
-* :ref:`modindex`
-* :ref:`search`

docs/prep-it.rst

@@ -0,0 +1,78 @@
+prep-it
+=======
+.. program:: prep-it
+
+The :program:`prep-it` utility scans the *platform* sub-directory and
+loads data files into various backends.  The following sections describe
+each directory name that is supported and its expected content.
+
+cassandra
+---------
+Text files containing queries that are executed using a
+:class:`cassandra.cluster.Cluster` instance.  Queries are separated by
+semi-colons and executed in the order that they appear in the file.  The
+Cassandra server is configured by setting the :envvar:`CASSANDRA_URI`
+environment variable.
+
+consul
+------
+JSON files containing top-level object definitions (e.g., dictionaries)
+that are loaded into a Consul key-value store using a :class:`consulate.Consul`
+instance.  The Consul endpoint is configured by setting the
+:envvar:`CONSUL_HOST` and :envvar:`CONSUL_PORT` environment variables.
+
+rabbitmq
+--------
+JSON files that contain RabbitMQ HTTP API commands to execute.  Each
+command is represented by an object with the following properties:
+
+:path:
+    The resource to send the request to.
+
+:method:
+    The HTTP method to invoke (e.g., ``POST``, ``DELETE``)
+
+:body:
+    The body to send with the request.
+
+The RabbitMQ server is identified by setting the :envvar:`RABBITMQ`
+environment variable to the host and port of the HTTP API endpoint.
+
+redis
+-----
+JSON files each containing a top-level object definition where each
+property names a redis command.  The property value is another object
+definition where the name is the redis key and the value is a list of
+values to pass to the command.
+
+For example, the following JSON file would result in calling the
+``SADD`` redis command to add ``"abuse"``, ``"admin"``, ``"postmaster"``,
+and ``"root"`` to the ``admin_type_address`` redis set.
+
+.. code-block:: javascript
+
+   {
+      "SADD": {
+         "admin_type_address": [
+            "abuse",
+            "admin",
+            "postmaster",
+            "root"
+         ]
+      }
+   }
+
+The redis server is configured by setting the :envvar:`REDIS_URI`
+environment variable to a `redis url`_.
+
+.. _redis url: https://www.iana.org/assignments/uri-schemes/prov/redis
+
+postgres
+--------
+SQL files that are executed using `queries`_.  The database server is
+configured by setting the :envvar:`PGSQL` environment variable.  The
+database name is based on the file name minus the assumed ``.sql``
+suffix.  The database will be dropped if it exists and then created
+anew before running the SQL commands from the file.
+
+.. _queries: https://github.com/gmr/queries