Merge branch '2.9' into stable

apx-parameters.rst

@@ -151,6 +151,8 @@ CVMFS_AVG_CHUNK_SIZE                | Desired Average size of a file chunk in by
                                     | (see also *CVMFS_USE_FILE_CHUNKING*)
 CVMFS_CATALOG_ALT_PATHS             | Enable/disable generation of catalog bootstrapping shortcuts during publishing.
                                     | (Useful when backend directory `/data` is not publicly accessible)
+CVMFS_CHECK_ALL_MIN_DAYS            | Minimum number of days between checking each repository with ``cvmfs_server check -a``
+                                    | Default value: 30
 CVMFS_COMPRESSION_ALGORITHM         | Compression algorithm to be used during publishing
                                     | (currently either 'default' or 'none')
 CVMFS_CREATOR_VERSION               | The CernVM-FS version that was used to create this repository

cpt-replica.rst

@@ -235,14 +235,35 @@ Monitoring
 The ``cvmfs_server`` utility reports status and problems to ``stdout``
 and ``stderr``.
 
-For the web server infrastructure, we recommend standard Nagios HTTP
-checks. They should be configured with the URL
-http://$replica-server/cvmfs/$repository_name/.cvmfspublished. This file
-can also be used to monitor if the same repository revision is served by
-the Stratum 0 server and all the Stratum 1 servers. In order to tune the
-hardware and cache sizes, keep an eye on the Squid server's CPU and I/O
-load.
+For the web server infrastructure, we recommend 
+`cvmfs-servermon <https://github.com/cvmfs-contrib/cvmfs-servermon>`
+which watches for problems in every repository's ``.cvmfs_status.json``
+status file.
+
+In order to tune the hardware and cache sizes, keep an eye on the Squid
+server's CPU and I/O load.
 
 Keep an eye on HTTP 404 errors. For normal CernVM-FS traffic, such
 failures should not occur. Traffic from CernVM-FS clients is marked by
 an ``X-CVMFS2`` header.
+
+.. _sct_stratum1_maintenance:
+
+Maintenance processes
+---------------------
+
+If any replicated repositories have Garbage Collection enabled, the
+Stratum 1 also needs to run garbage collection in order to prevent the
+disk space usage from growing rapidly.  Run ``cvmfs_server gc -af``
+periodically (e.g daily or weekly) from cron to run garbage collection
+on all repositories that have garbage collection enabled.  Logs will
+go into /var/log/cvmfs/gc.log. 
+
+In addition, over time problems can show up with a small percentage of
+files stored on a large Stratum 1.  Run ``cvmfs_server check -a`` daily
+from cron to start a check process.  On a large Stratum 1 it will run
+for many days, but only with a single thread so it is not very
+intrusive.  If another check is still in process a new one will not
+start.  Each repository by default will only be checked at most once
+every 30 days.  Logs will go into /var/log/cvmfs/checks.log and 
+problems will be recorded in a repository's ``.cvmfs_status.json``.

cpt-repo.rst

@@ -1388,10 +1388,9 @@ set ``CVMFS_AUTO_GC=true`` in the ``server.conf`` file of the stratum
 after a snapshot.  It will only work if the upstream stratum 0 repository has
 garbage collection enabled.
 
-Alternatively, ``cvmfs_server gc -af`` can be run from cron
-periodically (e.g. daily) to run garbage collection on all
-repositories that have garbage collection enabled on the stratum 0.
-Logs will go into /var/log/cvmfs/gc.log.
+Alternatively, all garbage collectable repositories can be automatically
+collected in turn separately from snapshots.
+See :ref:`sct_stratum1_maintenance`.
 
 Frequency of the Automatic Garbage Collection
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^