Merge remote-tracking branch 'upstream/develop' into develop

* upstream/develop: (128 commits)
  Fix up bad merge
  nix: Correct documentation for gc
  nix: Check for nix-collect-garbage
  Add nix module to Salt
  Inform on which python version we are running on
  Restore full user permissions when failing to delete
  Use six.moves to import cStringIO
  Remove unused imports
  salt.utils.gitfs: remove dulwich support, make refspecs configurable (saltstack#39210)
  Purged trailing whitespace
  Fix the new jinja_test failures (saltstack#39262)
  provide the possibility to put extra modules into the thin
  Add __utils__ to sdb unit test (saltstack#39261)
  support dryrun for dockerng.sls_build
  Pylint fix
  _device_mismatch_ignored will never be True
  Do boolean check instead len() to test whether string is empty
  salt/states/blockdev.py
  Remove blank line
  Add safety mechanism in shutdown fucntion and add passwd as valid argument
  ...

conf/master

@@ -623,10 +623,11 @@
 
 # Git File Server Backend Configuration
 #
-# Optional parameter used to specify the provider to be used for gitfs. Must
-# be one of the following: pygit2, gitpython, or dulwich. If unset, then each
-# will be tried in that same order, and the first one with a compatible
-# version installed will be the provider that is used.
+# Optional parameter used to specify the provider to be used for gitfs. Must be
+# either pygit2 or gitpython. If unset, then both will be tried (in that
+# order), and the first one with a compatible version installed will be the
+# provider that is used.
+#
 #gitfs_provider: pygit2
 
 # Along with gitfs_password, is used to authenticate to HTTPS remotes.
@@ -679,6 +680,11 @@
 # repository and defaults to the repository root.
 #gitfs_root: somefolder/otherfolder
 #
+# The refspecs fetched by gitfs remotes
+#gitfs_refspecs:
+#  - '+refs/heads/*:refs/remotes/origin/*'
+#  - '+refs/tags/*:refs/tags/*'
+#
 #
 #####         Pillar settings        #####
 ##########################################
@@ -828,6 +834,11 @@
 # to authenticate is protected by a passphrase.
 #git_pillar_passphrase: ''
 
+# The refspecs fetched by git_pillar remotes
+#git_pillar_refspecs:
+#  - '+refs/heads/*:refs/remotes/origin/*'
+#  - '+refs/tags/*:refs/tags/*'
+
 # A master can cache pillars locally to bypass the expense of having to render them
 # for each minion on every request. This feature should only be enabled in cases
 # where pillar rendering time is known to be unsatisfactory and any attendant security
@@ -1073,6 +1084,11 @@
 #winrepo_remotes:
 #  - 'https://github.com/saltstack/salt-winrepo.git'
 
+# The refspecs fetched by winrepo remotes
+#winrepo_refspecs:
+#  - '+refs/heads/*:refs/remotes/origin/*'
+#  - '+refs/tags/*:refs/tags/*'
+#
 
 #####      Returner settings          ######
 ############################################

doc/glossary.rst

@@ -1,3 +1,6 @@
+.. _glossary:
+
+========
 Glossary
 ========
 

doc/ref/clouds/all/salt.cloud.clouds.virtualbox.rst

@@ -1,6 +1,6 @@
-========================
+============================
 salt.cloud.clouds.virtualbox
-========================
+============================
 
 .. automodule:: salt.cloud.clouds.virtualbox
     :members:

doc/ref/configuration/examples.rst

@@ -1,3 +1,5 @@
+.. _configuration-file-examples:
+
 ===========================
 Configuration file examples
 ===========================

doc/ref/configuration/master.rst

@@ -245,6 +245,40 @@ each of Salt's module types such as ``runners``, ``output``, ``wheel``,
 
     extension_modules: /root/salt_extmods
 
+``extmod_whitelist``
+--------------------
+
+.. versionadded:: Nitrogen
+
+By using this dictionary, the modules that are synced to the master's extmod cache using `saltutil.sync_*` can be
+limited.  If nothing is set to a specific type, then all modules are accepted.  To block all modules of a specific type,
+whitelist an empty list.
+
+.. code-block:: yaml
+
+    extmod_whitelist:
+      modules:
+        - custom_module
+      engines:
+        - custom_engine
+      pillars: []
+
+Valid options:
+  - modules
+  - states
+  - grains
+  - renderers
+  - returners
+  - output
+  - proxy
+  - runners
+  - wheel
+  - engines
+  - queues
+  - pillar
+  - utils
+  - sdb
+
 .. conf_master:: module_dirs
 
 ``module_dirs``
@@ -760,7 +794,7 @@ Pass in an alternative location for the salt-ssh roster file.
 .. conf_master:: ssh_log_file
 
 ``ssh_log_file``
--------------------
+----------------
 
 .. versionadded:: 2016.3.5
 
@@ -1726,13 +1760,13 @@ Walkthrough <gitfs-per-remote-config>`.
 Optional parameter used to specify the provider to be used for gitfs. More
 information can be found in the :ref:`GitFS Walkthrough <gitfs-dependencies>`.
 
-Must be one of the following: ``pygit2``, ``gitpython``, or ``dulwich``. If
-unset, then each will be tried in that same order, and the first one with a
-compatible version installed will be the provider that is used.
+Must be either ``pygit2`` or ``gitpython``. If unset, then each will be tried
+in that same order, and the first one with a compatible version installed will
+be the provider that is used.
 
 .. code-block:: yaml
 
-    gitfs_provider: dulwich
+    gitfs_provider: gitpython
 
 .. conf_master:: gitfs_ssl_verify
 
@@ -2024,6 +2058,28 @@ authenticate is protected by a passphrase.
 
     gitfs_passphrase: mypassphrase
 
+.. conf_master:: gitfs_refspecs
+
+``gitfs_refspecs``
+~~~~~~~~~~~~~~~~~~
+
+.. versionadded:: Nitrogen
+
+Default: ``['+refs/heads/*:refs/remotes/origin/*', '+refs/tags/*:refs/tags/*']``
+
+When fetching from remote repositories, by default Salt will fetch branches and
+tags. This parameter can be used to override the default and specify
+alternate refspecs to be fetched. More information on how this feature works
+can be found in the :ref:`GitFS Walkthrough <gitfs-custom-refspecs>`.
+
+.. code-block:: yaml
+
+    gitfs_refspecs:
+      - '+refs/heads/*:refs/remotes/origin/*'
+      - '+refs/tags/*:refs/tags/*'
+      - '+refs/pull/*/head:refs/remotes/origin/pr/*'
+      - '+refs/pull/*/merge:refs/remotes/origin/merge/*'
+
 hg: Mercurial Remote File Server Backend
 ----------------------------------------
 
@@ -2959,7 +3015,7 @@ Git External Pillar Authentication Options
 ******************************************
 
 These parameters only currently apply to the ``pygit2``
-:conf_master:`git_pillar_provider`.  Authentication works the same as it does
+:conf_master:`git_pillar_provider`. Authentication works the same as it does
 in gitfs, as outlined in the :ref:`GitFS Walkthrough <gitfs-authentication>`,
 though the global configuration options are named differently to reflect that
 they are for git_pillar instead of gitfs.
@@ -3061,6 +3117,29 @@ authenticate is protected by a passphrase.
 
     git_pillar_passphrase: mypassphrase
 
+.. conf_master:: git_pillar_refspecs
+
+``git_pillar_refspecs``
+~~~~~~~~~~~~~~~~~~~~~~~
+
+.. versionadded:: Nitrogen
+
+Default: ``['+refs/heads/*:refs/remotes/origin/*', '+refs/tags/*:refs/tags/*']``
+
+When fetching from remote repositories, by default Salt will fetch branches and
+tags. This parameter can be used to override the default and specify
+alternate refspecs to be fetched. This parameter works similarly to its
+:ref:`GitFS counterpart <git_pillar-custom-refspecs>`, in that it can be
+configured both globally and for individual remotes.
+
+.. code-block:: yaml
+
+    git_pillar_refspecs:
+      - '+refs/heads/*:refs/remotes/origin/*'
+      - '+refs/tags/*:refs/tags/*'
+      - '+refs/pull/*/head:refs/remotes/origin/pr/*'
+      - '+refs/pull/*/merge:refs/remotes/origin/merge/*'
+
 .. _pillar-merging-opts:
 
 Pillar Merging Options
@@ -4028,3 +4107,26 @@ authenticate is protected by a passphrase.
 .. code-block:: yaml
 
     winrepo_passphrase: mypassphrase
+
+.. conf_master:: winrepo_refspecs
+
+``winrepo_refspecs``
+~~~~~~~~~~~~~~~~~~~~
+
+.. versionadded:: Nitrogen
+
+Default: ``['+refs/heads/*:refs/remotes/origin/*', '+refs/tags/*:refs/tags/*']``
+
+When fetching from remote repositories, by default Salt will fetch branches and
+tags. This parameter can be used to override the default and specify
+alternate refspecs to be fetched. This parameter works similarly to its
+:ref:`GitFS counterpart <winrepo-custom-refspecs>`, in that it can be
+configured both globally and for individual remotes.
+
+.. code-block:: yaml
+
+    winrepo_refspecs:
+      - '+refs/heads/*:refs/remotes/origin/*'
+      - '+refs/tags/*:refs/tags/*'
+      - '+refs/pull/*/head:refs/remotes/origin/pr/*'
+      - '+refs/pull/*/merge:refs/remotes/origin/merge/*'

doc/ref/configuration/minion.rst

@@ -1197,6 +1197,38 @@ below.
     providers:
       service: systemd
 
+``extmod_whitelist``
+--------------------
+
+.. versionadded:: Nitrogen
+
+By using this dictionary, the modules that are synced to the minion's extmod cache using `saltutil.sync_*`can be
+limited.  If nothing is set to a specific type, then all modules are accepted.  To block all modules of a specific type,
+whitelist an empty list.
+
+.. code-block:: yaml
+
+    extmod_whitelist:
+      modules:
+        - custom_module
+      engines:
+        - custom_engine
+      pillars: []
+
+Valid options:
+  - beacons
+  - sdb
+  - modules
+  - states
+  - grains
+  - renderers
+  - returners
+  - proxy
+  - engines
+  - output
+  - utils
+  - pillar
+
 
 Top File Settings
 =================
@@ -1416,6 +1448,10 @@ enabled and can be disabled by changing this value to ``False``.
 
     clean_dynamic_modules: True
 
+.. note::
+
+    If ``extmod_whitelist`` is specified, modules which are not whitelisted will also be cleaned here.
+
 .. conf_minion:: environment
 
 ``environment``

doc/ref/modules/all/index.rst

@@ -1,4 +1,4 @@
-.. _all-salt_modules:
+.. _all-salt.modules:
 
 =================
 execution modules

doc/ref/modules/all/salt.modules.cytest.rst

@@ -3,4 +3,3 @@ salt.modules.cytest module
 
 .. automodule:: salt.modules.cytest
     :members:
-

doc/ref/modules/all/salt.modules.nix.rst

@@ -0,0 +1,6 @@
+================
+salt.modules.nix
+================
+
+.. automodule:: salt.modules.nix
+    :members:

doc/ref/returners/index.rst

@@ -47,7 +47,7 @@ Writing a Returner
 
 A returner is a Python module containing at minimum a ``returner`` function.
 Other optional functions can be included to add support for
-:ref:`master_job_cache`, :ref:`external_job_cache`, and `Event Returners`_.
+:conf_master:`master_job_cache`, :ref:`external-job-cache`, and `Event Returners`_.
 
 ``returner``
     The ``returner`` function must accept a single argument. The argument
@@ -85,8 +85,8 @@ serializes the data as JSON and sets it in redis.
 Master Job Cache Support
 ------------------------
 
-:ref:`master_job_cache`, :ref:`external_job_cache`, and `Event Returners`_.
-Salt's :ref:`master_job_cache` allows returners to be used as a pluggable
+:conf_master:`master_job_cache`, :ref:`external-job-cache`, and `Event Returners`_.
+Salt's :conf_master:`master_job_cache` allows returners to be used as a pluggable
 replacement for the :ref:`default_job_cache`. In order to do so, a returner
 must implement the following functions:
 
@@ -176,7 +176,7 @@ must implement the following functions:
 External Job Cache Support
 --------------------------
 
-Salt's :ref:`external_job_cache` extends the :ref:`master_job_cache`. External
+Salt's :ref:`external-job-cache` extends the :conf_master:`master_job_cache`. External
 Job Cache support requires the following functions in addition to what is
 required for Master Job Cache support:
 
@@ -328,7 +328,7 @@ Testing the Returner
 
 The ``returner``, ``prep_jid``, ``save_load``, ``get_load``, and
 ``event_return`` functions can be tested by configuring the
-:ref:`master_job_cache` and `Event Returners`_ in the master config
+:conf_master:`master_job_cache` and `Event Returners`_ in the master config
 file and submitting a job to ``test.ping`` each minion from the master.
 
 Once you have successfully exercised the Master Job Cache functions, test the