Merge pull request #25412 from basepi/merge-forward-2015.8

[2015.8] Merge forward from 2015.5 to 2015.8

doc/_themes/saltstack2/layout.html

@@ -21,7 +21,7 @@
 
 {% set script_files = [
     '_static/js/core.min.js',
-    '_static/js/webhelp.min_v1.4.js',
+    '_static/js/webhelp.min_v1.4.1.js',
 ] %}
 
 {% set css_files = [

doc/_themes/saltstack2/static/js/webhelp.min_v1.4.js → doc/_themes/saltstack2/static/js/webhelp.min_v1.4.1.js

@@ -11,7 +11,7 @@ $( document ).ready(function() {
 
     /*insert links to module functions*/
     if ($( 'a.current' ).length && $( 'dt .headerlink' ).length ) {
-        if (!window.location.href.indexOf('/ref/cli/')) {
+        if (window.location.href.indexOf('/ref/cli/') == -1) {
             var tgt = $( 'a.current' );
             tgt.after('<ul id="function-list"></ul>');
             $('dt .headerlink').each(function(idx, elem) {

doc/conf.py

@@ -162,7 +162,7 @@ def __getattr__(cls, name):
 copyright = '2015 SaltStack, Inc.'
 
 version = salt.version.__version__
-latest_release = '2015.5.2'  # latest release
+latest_release = '2015.5.3' # latest release
 previous_release = '2014.7.6'  # latest release from previous branch
 previous_release_dir = '2014.7'  # path on web server for previous branch
 build_type = 'develop'  # latest, previous, develop

doc/contents.rst

@@ -17,6 +17,7 @@ Salt Table of Contents
     topics/eauth/access_control
     topics/jobs/index
     topics/jobs/job_cache
+    topics/jobs/external_cache
     topics/sdb/index
     topics/event/index
     topics/beacons/index

doc/ref/clientacl.rst

@@ -13,13 +13,15 @@ specified user. This configuration is much like the ``peer`` configuration:
 
 .. code-block:: yaml
 
-    # Allow thatch to execute anything and allow fred to use ping and pkg
     client_acl:
+      # Allow thatch to execute anything.
       thatch:
         - .*
+      # Allow fred to use test and pkg, but only on "web*" minions.
       fred:
-        - test.*
-        - pkg.*
+        - web*:
+          - test.*
+          - pkg.*
 
 Permission Issues
 =================
@@ -45,4 +47,4 @@ existing user keys and re-start the Salt master:
 .. code-block:: bash
 
     rm /var/cache/salt/.*key
-    service salt-master restart
+    service salt-master restart

doc/ref/states/requisites.rst

@@ -47,7 +47,7 @@ Requisites come in two types: Direct requisites (such as ``require``),
 and requisite_ins (such as ``require_in``). The relationships are
 directional: a direct requisite requires something from another state.
 However, a requisite_in inserts a requisite into the targeted state pointing to
-the targeting state.  The following example demonstrates a direct requisite:
+the targeting state. The following example demonstrates a direct requisite:
 
 .. code-block:: yaml
 
@@ -463,8 +463,8 @@ the specified commands return ``False``. The ``unless`` requisite operates
 as NOR and is useful in giving more granular control over when a state should
 execute.
 
-**NOTE**: Under the hood ``unless`` calls ``cmd.retcode`` with 
-``python_shell=True``.  This means the commands referenced by unless will be
+**NOTE**: Under the hood ``unless`` calls ``cmd.retcode`` with
+``python_shell=True``. This means the commands referenced by unless will be
 parsed by a shell, so beware of side-effects as this shell will be run with the
 same privileges as the salt-minion.
 
@@ -494,7 +494,7 @@ For example:
         - names:
           - first_deploy_cmd
           - second_deploy_cmd
-        - unless: some_check
+        - unless: ls /usr/bin/vim
 
 In the above case, ``some_check`` will be run prior to _each_ name -- once for
 ``first_deploy_cmd`` and a second time for ``second_deploy_cmd``.
@@ -508,8 +508,8 @@ Onlyif
 return ``True``, then the state is run. If any of the specified commands
 return ``False``, the state will not run.
 
-**NOTE**: Under the hood ``onlyif`` calls ``cmd.retcode`` with 
-``python_shell=True``.  This means the commands referenced by unless will be
+**NOTE**: Under the hood ``onlyif`` calls ``cmd.retcode`` with
+``python_shell=True``. This means the commands referenced by unless will be
 parsed by a shell, so beware of side-effects as this shell will be run with the
 same privileges as the salt-minion.
 
@@ -587,8 +587,8 @@ check_cmd
 Check Command is used for determining that a state did or did not run as
 expected.
 
-**NOTE**: Under the hood ``check_cmd`` calls ``cmd.retcode`` with 
-``python_shell=True``.  This means the commands referenced by unless will be
+**NOTE**: Under the hood ``check_cmd`` calls ``cmd.retcode`` with
+``python_shell=True``. This means the commands referenced by unless will be
 parsed by a shell, so beware of side-effects as this shell will be run with the
 same privileges as the salt-minion.
 

doc/topics/jobs/external_cache.rst

@@ -0,0 +1,163 @@
+.. _external-master-cache:
+
+=========================================
+Storing Job Results in an External System
+=========================================
+
+After a job executes, job results are returned
+to the Salt Master by each Salt Minion. These results are stored in the
+:ref:`Default Job Cache <default_job_cache>`.
+
+In addition to the Default Job Cache, Salt provides two additional
+mechanisms to send job results to other systems (databases, local syslog,
+and others):
+
+* External Job Cache
+* Master Job Cache 
+
+The major difference between these two
+mechanism is from where results are returned (from the Salt Master or Salt
+Minion).
+
+External Job Cache - Minion-Side Returner
+-----------------------------------------
+
+When an External Job Cache is configured, data is returned to the Default Job
+Cache on the Salt Master like usual, and then results are also sent to an
+External Job Cache using a Salt returner module running on the Salt Minion.
+
+.. image:: /_static/external-job-cache.png
+    :align: center
+
+* Advantages: Data is stored without placing additional load on the Salt Master.
+* Disadvantages: Each Salt Minion connects to the external job cache, which can
+  result in a large number of connections.  Also requires additional configuration to
+  get returner module settings on all Salt Minions.
+
+Master Job Cache - Master-Side Returner
+---------------------------------------
+
+.. versionadded:: 2014.7.0
+
+Instead of configuring an External Job Cache on each Salt Minion, you can
+configure the Master Job Cache to send job results from the Salt Master
+instead. In this configuration, Salt Minions send data to the Default Job Cache
+as usual, and then the Salt Master sends the data to the external system using
+a Salt returner module running on the Salt Master.
+
+.. image:: /_static/master-job-cache.png
+    :align: center
+
+* Advantages: A single connection is required to the external system. This is
+  preferred for databases and similar systems.
+* Disadvantages: Places additional load on your Salt Master.
+
+Configure an External or Master Job Cache
+-----------------------------------------
+
+Step 1: Understand Salt Returners
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Before you configure a job cache, it is essential to understand Salt returner
+modules ("returners"). Returners are pluggable Salt Modules that take the data
+returned by jobs, and then perform any necessary steps to send the data to an
+external system. For example, a returner might establish a connection,
+authenticate, and then format and transfer data. 
+
+The Salt Returner system provides the core functionality used by the External
+and Master Job Cache systems, and the same returners are used by both systems.
+
+Salt currently provides many different returners that let you connect to a
+wide variety of systems. A complete list is available at
+:ref:`all Salt returners <all-salt.returners>`.
+Each returner is configured differently, so make sure you read and follow the
+instructions linked from that page.
+
+For example, the MySQL returner requires:
+
+* A database created using provided schema (structure is available at
+  :mod:`MySQL returner <salt.returners.mysql>`)
+* A user created with with privileges to the database
+* Optional SSL configuration
+
+A simpler returner, such as Slack or HipChat, requires:
+
+* An API key/version
+* The target channel/room
+* The username that should be used to send the message
+
+Step 2: Configure the Returner
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+After you understand the configuration and have the external system ready, add
+the returner configuration settings to the Salt Minion configuration file for
+the External Job Cache, or to the Salt Master configuration file for the Master
+Job Cache. 
+
+For example, MySQL requires:
+
+.. code-block:: yaml
+
+    mysql.host: 'salt'
+    mysql.user: 'salt'
+    mysql.pass: 'salt'
+    mysql.db: 'salt'
+    mysql.port: 3306
+
+Slack requires:
+
+.. code-block:: yaml
+
+    slack.channel: 'channel' 
+    slack.api_key: 'key'
+    slack.from_name: 'name'
+
+After you have configured the returner and added settings to the configuration
+file, you can enable the External or Master Job Cache.
+
+Step 3: Enable the External or Master Job Cache
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Configuration is a single line that specifies an
+already-configured returner to use to send all job data to an external system.
+
+External Job Cache
+""""""""""""""""""
+
+To enable a returner as the External Job Cache (Minion-side), add the following
+line to the Salt Master configuration file:
+
+.. code-block:: yaml
+
+    ext_job_cache: <returner>
+
+For example:
+
+.. code-block:: yaml
+
+    ext_job_cache: mysql
+
+.. note::
+    When configuring an External Job Cache (Minion-side), the returner settings are
+    added to the Minion configuration file, but the External Job Cache setting
+    is configured in the Master configuration file.
+
+Master Job Cache
+""""""""""""""""
+
+To enable a returner as a Master Job Cache (Master-side), add the following
+line to the Salt Master configuration file:
+
+.. code-block:: yaml
+
+    master_job_cache: <returner>
+
+For example:
+
+.. code-block:: yaml
+
+    master_job_cache: mysql
+
+Verify that the returner configuration settings are in the Master configuration
+file, and be sure to restart the salt-master service after you make
+configuration changes. (``service salt-master restart``).

doc/topics/jobs/job_cache.rst

@@ -1,10 +1,9 @@
-=======================
+======================
 Managing the Job Cache
-=======================
+======================
 
 The Salt Master maintains a job cache of all job executions which can be
-queried via the jobs runner. The way this job cache is managed is very
-pluggable via Salt's underlying returner interface.
+queried via the jobs runner. This job cache is called the Default Job Cache.
 
 .. _default_job_cache:
 
@@ -14,7 +13,7 @@ Default Job Cache
 A number of options are available when configuring the job cache. The default
 caching system uses local storage on the Salt Master and can be found in the
 job cache directory (on Linux systems this is typically
-/var/cache/salt/master/jobs). The default caching system is suitable for most
+``/var/cache/salt/master/jobs``). The default caching system is suitable for most
 deployments as it does not typically require any further configuration or
 management.
 
@@ -28,47 +27,13 @@ Salt Master configuration file. The value passed in is measured via hours:
 
     keep_jobs: 24
 
-External Job Cache Options
-==========================
+Additional Job Cache Options
+============================
 
 Many deployments may wish to use an external database to maintain a long term
 register of executed jobs. Salt comes with two main mechanisms to do this, the
-master job cache and the external job cache. The difference is how the external
-data store is accessed.
+master job cache and the external job cache.
 
-.. _master_job_cache:
+See :ref:`Storing Job Results in an External System <external-master-cache>`.
 
-Master Job Cache
-================
 
-.. versionadded:: 2014.7
-
-The master job cache setting makes the built in job cache on the master
-modular. This system allows for the default cache to be swapped out by the Salt
-returner system. To configure the master job cache, set up an external returner
-database based on the instructions included with each returner and then simply
-add the following configuration to the master configuration file:
-
-.. code-block:: yaml
-
-    master_job_cache: mysql
-
-.. _external_job_cache:
-
-External Job Cache
-==================
-
-The external job cache setting instructs the minions to directly contact the
-data store. This scenario is helpful when the data store needs to be made
-available to the minions. This can be an effective way to share historic data
-across an infrastructure as data can be retrieved from the external job cache
-via the ``ret`` execution module.
-
-To configure the external job cache, set up a returner database in the manner
-described in the specific returner documentation. Ensure that the returner
-database is accessible from the minions, and set the `ext_job_cache` setting
-in the master configuration file:
-
-.. code-block:: yaml
-
-    ext_job_cache: redis