docs: Added automatic local tables of content

Until now, we had to write the tables of content for individual book pages by hand. This was annoying and error-prone, since changes to the document structure would not necessarily be reflected in the TOC. Additionally, many rendering issues could be observed in the output, most of which were due to incorrect and inconsistent markdown lists. The markdown-to-HTML converter we use offers a TOC plugin for exactly that purpose (for documentation and use, see the `TOC` section in https://github.com/walokra/markdown-page-generator-plugin). This patch enables the plugin and converts the pages in the Configuration section of the Book to use it. The only TOCs that will still need to be manually edited are those in the chaper heading pages; there is (yet) no mechanism to do this automatically.
dCache · Aug 6, 2019 · a216f89 · a216f89
1 parent 6925c44
commit a216f89
Show file tree

Hide file tree

Showing 38 changed files with 137 additions and 371 deletions.
diff --git a/docs/TheBook/pom.xml b/docs/TheBook/pom.xml
@@ -42,7 +42,7 @@
             <headerHtmlFile>${html-header.path}</headerHtmlFile>
             <footerHtmlFile>${html-footer.path}</footerHtmlFile>
             <transformRelativeMarkdownLinks>true</transformRelativeMarkdownLinks>
-            <pegdownExtensions>SMARTYPANTS,AUTOLINKS,TABLES,FENCED_CODE_BLOCKS,STRIKETHROUGH,TASKLISTITEMS,EXTANCHORLINKS</pegdownExtensions>
+            <pegdownExtensions>SMARTYPANTS,AUTOLINKS,TABLES,FENCED_CODE_BLOCKS,STRIKETHROUGH,TASKLISTITEMS,EXTANCHORLINKS,TOC</pegdownExtensions>
             <copyDirectories>css,images</copyDirectories>
           </configuration>
       </plugin>

diff --git a/docs/TheBook/src/main/markdown/config-PoolManager.md b/docs/TheBook/src/main/markdown/config-PoolManager.md
@@ -1,22 +1,6 @@
 THE POOLMANAGER SERVICE
 ==================================
 
-## Table of Contents
-
-* [The Pool Selection Mechanism](#the-pool-selection-mechanism)
-
-  * [Links](#links)
-  * [Examples](#examples)
-
-* [The Partition Manager](#the-partition-manager)
-
-  * [Overview](#overview)
-  * [Managing Partitions](#managing-partitions)
-  * [Using Partitions](#using-partitions)
-  * [Classic Partitions](#classic-partitions)
-
-* [Link Groups](#link-groups)
-
 The heart of a dCache System is the `poolmanager`. When a user performs an action on a file - reading or writing - a `transfer request` is sent to the dCache system. The `poolmanager` then decides how to handle this request.
 
 If a file the user wishes to read resides on one of the storage-pools within the dCache system, it will be transferred from that pool to the user. If it resides on several pools, the file will be retrieved from one of the pools determined by a configurable load balancing policy. If all pools the file is stored on are busy, a new copy of the file on an idle pool will be created and this pool will answer the request.
@@ -42,6 +26,10 @@ chapter we will describe the commands allowed in this file.
 > `poolmanager` in zookeeper and reads `poolmanager.conf` file only if
 > configuration in zookeeper is missing, e.g. on the first start.
 
+-----
+[TOC bullet hierarchy]
+-----
+
 ## THE POOL SELECTION MECHANISM
 
 The PSU is responsible for finding the set of pools which can be used for a specific transfer-request. By telling the PSU which pools are permitted for which type of transfer-request, the administrator of the dCache system can adjust the system to any kind of scenario: Separate organizations served by separate pools, special pools for writing the data to a tertiary storage system, pools in a DMZ which serves only a certain kind of data (e.g., for the grid). This section explains the mechanism employed by the PSU and shows how to configure it with several examples.

diff --git a/docs/TheBook/src/main/markdown/config-SRM.md b/docs/TheBook/src/main/markdown/config-SRM.md
@@ -1,43 +1,6 @@
 CHAPTER 13. dCache STORAGE RESOURCE MANAGER
 ===========================================
 
-## Table of Contents
-
-+ [Introduction](#introduction)
-+ [Configuring the srm service](#configuring-the-srm-service)
-
-	 [The Basic Setup](#the-basic-setup)
-	 [Important srm configuration options](#important-srm-configuration-options)
-
-+ [Utilization of Space Reservations for Data Storage](#utilization-of-space-reservations-for-data-storage)
-
-	 [Properties of Space Reservation](#properties-of-space-reservation)
-+ [dCache specific concepts](#dcache-specific-concepts)
-
-	 [Activating SRM SpaceManager](#activating-srm-spacemanager)
-	 [Explicit and Implicit Space Reservations for Data Storage in dCache](#explicit-and-implicit-space-reservations-for-data-storage-in-dcache)
-
-+ [SpaceManager configuration](#spacemanager-configuration)
-
-	 [SRM SpaceManager and Link Groups](#srm-spacemanager-and-link-groups)
-         [Making a Space Reservation](#making-a-space-reservation)
-	 [SRM configuration for experts](#srm-configuration-for-experts)
-
-+ [Configuring the PostgreSQL Database](#configuring-the-postgresql-database)
-
-	 [SRM or srm monitoring on a separate node](#srm-or-srm-monitoring-on-a-separate-node)
-
-+ [General SRM Concepts (for developers)](#general-srm-concepts-(for-developers))
-
-	 [The SRM service](#the-srm-service)
-	 [Space Management Functions](#space-management-functions)
-	 [Data Transfer Functions](#data-transfer-functions)
-	 [Request Status Functions](#request-status-functions)
-	 [Directory Functions](#directory-functions)
-	 [Permission functions](#permission-functions)
-
-## Introduction
-
 Storage Resource Managers (SRMs) are middleware components whose function is to
 provide dynamic space allocation and file management on shared storage components on the Grid.
 SRMs support protocol negotiation and a reliable replication mechanism.
@@ -55,6 +18,10 @@ The main benefits of using HTTPS rather than HTTP over GSI is that HTTPS is a st
 and has support for sessions, improving latency in case a client needs to connect to the
 same server multiple times.
 
+-----
+[TOC bullet hierarchy]
+-----
+
 ## CONFIGURING THE SRM SERVICE
 
 ### BASIC SETUP

diff --git a/docs/TheBook/src/main/markdown/config-acl.md b/docs/TheBook/src/main/markdown/config-acl.md
@@ -1,18 +1,14 @@
 ACLs in dCache
 ===========================
 
-Table of Contents
------------------
-
-* [Introduction](#introduction)
-* [Configuring ACL support](#configuring-acl-support)
-* [Setting and getting ACLs](#setting-and-getting-acls)
-* [Accessing ACL over NFS mount](#accessing-acl-over-nfs-mount)
-
 dCache includes support for Access Control Lists (ACLs). This support is conforming to the [NFS version 4 Protocol specification](https://tools.ietf.org/rfc/rfc7530.txt).
 
 This chapter provides some background information and details on configuring dCache to use ACLs and how to administer the resulting system.
 
+-----
+[TOC bullet hierarchy]
+-----
+
 ## Introduction
 
 dCache allows control over namespace operations (e.g., creating new files and directories, deleting items, renaming items) and data operations (reading data, writing data) using the standard Unix permission model. In this model, files and directories have both owner and group-owner attributes and a set of permissions that apply to the owner, permissions for users that are members of the group-owner group and permissions for other users.

diff --git a/docs/TheBook/src/main/markdown/config-admin.md b/docs/TheBook/src/main/markdown/config-admin.md
@@ -5,6 +5,10 @@ THE ADMIN SERVICE
 >
 > Only commands described in this documentation should be used for the administration of a dCache system.
 
+-----
+[TOC bullet hierarchy]
+-----
+
 ## FIRST STEPS
 
 dCache has a powerful administration interface.  Administration protocol is implemented as `admin` cell that

diff --git a/docs/TheBook/src/main/markdown/config-alarms.md b/docs/TheBook/src/main/markdown/config-alarms.md
@@ -1,20 +1,6 @@
 CHAPTER 16. THE ALARMS SERVICE
 ==============================
 
-## Table of Contents
-
-* [The Basic Setup](#the-basic-setup)
-      [Configure where the alarms service is Running](#configure-where-the-alarms-service-is-running)
-      [Types of Alarms](#types-of-alarms)
-      [Alarm Priority](#alarm-priority)
-      [Working with Alarms: Shell Commands](#working-with-alarms-shell-commands)
-      [Working with Alarms: Admin Commands](#working-with-alarms-admin-commands)
-      [Working with Alarms: The DCache-View Alarms Tab](#working-with-alarms-the-dcache-view-alarms-tab)
-* [Advanced Service Configuration: Enabling Automatic Cleanup](#advanced-service-configuration-enabling-automatic-cleanup)
-* [Advanced Service Configuration: Enabling Email Alerts](#advanced-service-configuration-enabling-email-alerts)
-* [Miscellaneous Properties of the Alarms Service](#miscellaneous-properties-of-the-alarm-service)
-* [Alarms SPI](#alarms-spi--service-provider-interface-)
-
 dCache has an `alarms` backend service which records failures (*alarms*)
 requiring more or less urgent intervention. The service stores alarms by
 either in an XML file or an RDBMS. The service is turned off by default.
@@ -26,6 +12,10 @@ both the timestamp range and the filtering and sorting of alarms by fields.
 Admins also have access to buttons which allow them to
 mark alarms as closed or to delete them altogether.
 
+-----
+[TOC bullet hierarchy]
+-----
+
 ## THE BASIC SETUP
 
 It is not necessary to run the `alarms` service in a separate domain, though

diff --git a/docs/TheBook/src/main/markdown/config-billing.md b/docs/TheBook/src/main/markdown/config-billing.md
@@ -1,14 +1,6 @@
 CHAPTER 15. THE BILLING SERVICE
 ===============================
 
-## Table of Contents
-
-* [The Billing Log Files](#the-billing-log-files)
-* [The Billing Database](#the-billing-database)
-* [Billing Histogram Data](#billing-histogram-data)
-* [Billing Records](#billing-records)
-* [Upgrading a Previous Installation](#upgrading-a-previous-installation)
-
 dCache has built-in monitoring capabilities which provide an overview of the activity and performance of the installation’s doors and pools. There are two options for how this data can be represented and stored:
 
 -   a set of log files written to a known location
@@ -18,6 +10,10 @@ dCache has built-in monitoring capabilities which provide an overview of the act
 
 These options can be enabled simultaneously. If the database option is selected, the data in those tables will also be displayed as a set of histogram plots on the installation's web page.
 
+-----
+[TOC bullet hierarchy]
+-----
+
 ## THE BILLING LOG FILES
 
 If you installed dCache following the instructions in the Chapter [Installing dCache](install.md) you enabled the BILLING in the domain where the HTTPD service is running (see the extract of the layout file).

diff --git a/docs/TheBook/src/main/markdown/config-chimera.md b/docs/TheBook/src/main/markdown/config-chimera.md
@@ -1,18 +1,9 @@
 CHIMERA
 ==================
 
-## Table of Contents
-
-- [Mounting Chimera through NFS](#mounting-chimera-through-nfs)
-- [Using dCap with a mounted file system](#using-dcap-with-a-mounted-file-system)
-- [Communicating with Chimera](#communicating-with-chimera)
-- [IDs](#ids)
-- [Directory Tags](#directory-tags)
-    - [Create, List and Read Directory Tags if the Namespace is not Mounted](#create-list-and-read-directory-tags-if-the-namespace-is-not-mounted)
-    - [Create, List and Read Directory Tags if the Namespace is Mounted](#create-list-and-read-directory-tags-if-the-namespace-is-mounted)
-    - [Directory Tags and Command Files](#directory-tags-and-command-files)
-    - [Directory Tags for dCache](#directory-tags-for-dcache)
-    - [Storage Class and Directory Tags](#storage-class-and-directory-tags)
+-----
+[TOC bullet hierarchy]
+-----
 
 The inner dCache components talk to the namespace via a module called `PnfsManager`, which in turn communicates with the Chimera database using a thin Java layer. In addition to `PnfsManager` a direct access to the file system view is provided by an `NFSv3` and `NFSv4.1` server. Clients can `NFS`-mount the namespace locally. This offers the opportunity to use OS-level tools like `ls, mkdir, mv` for Chimera. Direct I/O-operations like `cp` and `cat` are possible with the `NFSv4.1 door`.
 

diff --git a/docs/TheBook/src/main/markdown/config-frontend.md b/docs/TheBook/src/main/markdown/config-frontend.md
@@ -1,5 +1,9 @@
 CHAPTER 17. dCache Frontend Service
-=====================================
+===================================
+
+-----
+[TOC bullet hierarchy]
+-----
 
 The Frontend service is the dCache service (cell) responsible for
 serving data to clients via HTTP/REST.  The default port on which it

diff --git a/docs/TheBook/src/main/markdown/config-ftp.md b/docs/TheBook/src/main/markdown/config-ftp.md
@@ -1,16 +1,13 @@
 dCache as an FTP Server
 =======================
 
-Table of Contents
-
-- [Introduction](#introduction)
-- [The Control Channel](#the-control-channel)
-- [Data transfers](#data-transfers)
-- [Configuration examples](#configuration-examples)
-
 This chapter explains how to configure dCache to allow FTP, a common
 network protocol that many clients support.
 
+-----
+[TOC bullet hierarchy]
+-----
+
 ## Introduction
 
 FTP is a long established protocol that allows clients to transfer

diff --git a/docs/TheBook/src/main/markdown/config-gplazma.md b/docs/TheBook/src/main/markdown/config-gplazma.md
@@ -1,37 +1,12 @@
 Chapter 10. Authorization in dCache
 ===================================
 
-## Table of Contents
-
-+ [Basics](#basics)
-+ [Configuration](#configuration)
-
-    - [Plug-ins](#plug-ins)
-
-+ [Using X.509 Certificates](#using-x509-certificates)
-
-    - [CA Certificates](#ca-certificates)
-    - [User Certificate](#user-certificate)
-    - [Host Certificate](#host-certificate)
-    - [VOMS Proxy Certificate](#voms-proxy-certificate)
-
-+ [Using OpenID Connect](#using-openid-connect)
-
-+ [Configuration files](#configuration-files)
-
-    - [storage-authzdb](#storage-authzdb)
-    - [The gplazmalite-vorole-mapping plug-in](#the-gplazmalite-vorole-mapping-plug-in)
-    - [Authorizing a VO](#authorizing-a-vo)
-    - [The kpwd plug-in](#the-kpwd-plug-in)
-    - [The gridmap plug-in](#the-gridmap-plug-in)
-
-+ [gPlazma specific dCache configuration](#gplazma-specific-dcache-configuration)
-
-    - [Enabling Username/Password Access for WebDAV](#enabling-username-password-access-for-webdav)
-    - [gPlazma config example to work with authenticated webadmin](#gplazma-config-example-to-work-with-authenticated-webadmin)
-
 To limit access to data, dCache comes with an authentication and authorization interface called `gPlazma2`. gPlazma is an acronym for Grid-aware PLuggable AuthorZation Management. Earlier versions of dCache worked with `gPlazma1` which has now been completely removed from dCache. So if you are upgrading, you have to reconfigure `gPlazma` if you used `gPlazma1` until now.
 
+-----
+[TOC bullet hierarchy]
+-----
+
 ## Basics
 
 Though it is possible to allow anonymous access to dCache it is usually desirable to authenticate users. The user then has to connect to one of the different doors (e.g., `GridFTP door, dCap door`) and login with credentials that prove his identity. In Grid-World these credentials are very often `X.509` certificates, but dCache also supports other methods like username/password and kerberos authentication.

diff --git a/docs/TheBook/src/main/markdown/config-history.md b/docs/TheBook/src/main/markdown/config-history.md
@@ -1,6 +1,10 @@
 CHAPTER 18. dCache History Service
 =====================================
 
+-----
+[TOC bullet hierarchy]
+-----
+
 The purpose of this service is to provide a disk-backed cache for time-windowed
 state data extracted from backend dCache components, most importantly, pools.
 

diff --git a/docs/TheBook/src/main/markdown/config-hopping.md b/docs/TheBook/src/main/markdown/config-hopping.md
@@ -1,15 +1,9 @@
 CHAPTER 9. FILE HOPPING
 =======================
 
-
-## Table of Contents
-
-* [File Hopping on arrival from outside dCache](#file-hopping-on-arrival-from-outside-dcache)
-
-    [File mode of replicated files](#file-mode-of-replicated-files)
-    [File Hopping managed by the PoolManager](#file-hopping-managed-by-the-poolmanager)
-    [File Hopping managed by the HoppingManager](#file-hopping-managed-by-the-hoppingmanager)
-
+-----
+[TOC bullet hierarchy]
+-----
 
 File hopping is a collective term in dCache, summarizing the possibility of having files being transferred between dCache pools triggered by a variety of conditions. The most prominent examples are:
 

diff --git a/docs/TheBook/src/main/markdown/config-hsm.md b/docs/TheBook/src/main/markdown/config-hsm.md
@@ -1,43 +1,12 @@
 Chapter 8: The dCache Tertiary Storage System Interface
 ============================================
 
-
-Table of Contents
-
-* [Introduction](#introduction)
-* [Scope of this chapter](#scope-of-this-chapter)
-* [Requirements for a Tertiary Storage System](#requirements-for-a-tertiary-storage-system)
-
-    [Migrating Tertiary Storage Systems with a file system interface.](#migrating-tertiary-storage-systems-with-a-file-system-interface.)
-    [Tertiary Storage Systems with a minimalistic PUT, GET and REMOVE interface](#tertiary-storage-systems-with-a-minimalistic-put-get-and-remove-interface)
-
-* [How dCache interacts with a Tertiary Storage System](#how-dcache-interacts-with-a-tertiary-storage-system)
-* [Details on the TSS-support executable](#details-on-the-tss-support-executable)
-
-     [Summary of command line options](#summary-of-command-line-options)
-     [Summary of return codes](#summary-of-return-codes)
-     [The executable and the STORE FILE operation](#the-executable-and-the-store-file-operation)
-     [The executable and the FETCH FILE operation](#the-executable-and-the-fetch-file-operation)
-     [The executable and the REMOVE FILE operation](#the-executable-and-the-remove-file-operation)
-
-* [Configuring pools to interact with a Tertiary Storage System](#configuring-pools-to-interact-with-a-tertiary-storage-system)
-
-     [The dCache layout files](#the-dcache-layout-files)
-     [What happens next](#what-happens-next)
-
-* [How to Store-/Restore files via the Admin Interface](#how-to-store-/restore-files-via-the-admin-interface)
-* [How to monitor what’s going on](#how-to-monitor-what’s-going-on)
-
-     [Log Files](#log-files)
-     [Obtain information via the dCache Command Line Admin Interface](#obtain-information-via-the-dcache-command-line-admin-interface)
-
-* [Example of an executable to simulate a tape backend](#example-of-an-executable-to-simulate-a-tape-backend)
-
-
-## INTRODUCTION
-
 One of the features dCache provides is the ability to migrate files from its disk repository to one or more connected Tertiary Storage Systems (TSS) and to move them back to disk when necessary. Although the interface between dCache and the TSS is kept simple, dCache assumes to interact with an intelligent TSS. dCache does not drive tape robots or tape drives by itself. More detailed requirements to the storage system are described in one of the subsequent paragraphs.
 
+-----
+[TOC bullet hierarchy]
+-----
+
 ## SCOPE OF THIS CHAPTER
 
 This document describes how to enable a standard dCache installation to interact with a Tertiary Storage System. In this description we assume that

diff --git a/docs/TheBook/src/main/markdown/config-info-provider.md b/docs/TheBook/src/main/markdown/config-info-provider.md
@@ -1,15 +1,6 @@
 CHAPTER 19. GLUE INFO PROVIDER
 ==============================
 
-## Table of Contents
-+ [Internal collection of information](#internal-collection-of-information)
-+ [Configuring the info service](#configuring-the-info-service)
-+ [Testing the info provider](#testing-the-info-provider)
-+ [Publishing dCache information](#publishing-dcache-information)
-+ [Troubleshooting BDII problems](#troubleshooting-bdii-problems)
-+ [Updating information](#updating-information)
-
-
 The GLUE information provider supplied with dCache provides the information about the dCache instance in a standard format called GLUE. This is necessary so that WLCG infrastructure (such as FTS) and clients using WLCG tools can discover the dCache instance and use it correctly.
 
 The process of configuring the info-provider is designed to have the minimum overhead so you can configure it manually; however, you may prefer to use an automatic configuration tool, such as YAIM.
@@ -24,6 +15,10 @@ This chapter describes how to enable and test the dCache-internal collection of
 >
 > Please be aware that changing information provider may result in a brief interruption to published information. This may have an adverse affect on client software that make use of this information.
 
+-----
+[TOC bullet hierarchy]
+-----
+
 ## INTERNAL COLLECTION OF INFORMATION
 
 The info-provider takes as much information as possible from dCache. To achieve this, it needs the internal information-collecting service, `info`, to be running and a means to collect that information: `httpd`. Make sure that both the `httpd` and `info` services are running within your dCache instance. By default, the `info` service is started on the admin-node; but it is possible to configure dCache so it runs on a different node. You should run only one `info` service per dCache instance.