Configuring Read the Docs to publish the MPS manual

.readthedocs.yaml

@@ -0,0 +1,69 @@
+# .readthedocs.yaml -- Build configuration for MPS manual on Read The Docs
+#
+# Copyright (c) 2023 Ravenbrook Limited. See end of file for license.
+#
+# This file controls how Read the Docs builds and publishes the MPS
+# manual at <https://memory-pool-system.readthedocs.io/>.
+#
+# See <https://docs.readthedocs.io/en/stable/config-file/v2.html> for
+# the file format.
+#
+# Project configuration is at
+# <https://readthedocs.org/projects/memory-pool-system/>.  The GitHub
+# Ravenbot user <sysadmins@ravenbrook.com> is an administrator
+# <https://readthedocs.org/profiles/Ravenbot/>.
+
+version: 2
+
+build:
+  os: ubuntu-22.04
+  tools:
+    python: '3'
+
+python:
+  install:
+    - requirements: manual/requirements.pip
+
+sphinx:
+  configuration: manual/source/conf.py
+
+# A. REFERENCES
+#
+# [Readthedocs] "Read the Docs: Documentation Simplified";
+# <https://docs.readthedocs.io/en/stable/>.
+#
+#
+# B. DOCUMENT HISTORY
+#
+# 2023-02-02 RB Created as part of MPS GitHub migration.
+#
+#
+# C. COPYRIGHT AND LICENSE
+#
+# Copyright © 2023 Ravenbrook Limited <https://www.ravenbrook.com/>.
+# 
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met:
+# 
+# 1. Redistributions of source code must retain the above copyright
+#    notice, this list of conditions and the following disclaimer.
+# 
+# 2. Redistributions in binary form must reproduce the above copyright
+#    notice, this list of conditions and the following disclaimer in the
+#    documentation and/or other materials provided with the distribution.
+# 
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+#
+#
+# $Id$

design/doc.txt

@@ -93,11 +93,14 @@ _`.impl.design`: Design documents are written in plain RST (with no
 custom directives) to meet `.req.design.standalone`_.
 
 _`.impl.design.pelican`: Design documents are converted to HTML using
-|RstReader|_.
+|RstReader|_ as part of `Charlotte
+<https://info.ravenbrook.com/project/charlotte>`__.
 
 .. |RstReader| replace:: ``pelican.readers.RstReader``
 .. _RstReader: https://fossies.org/dox/pelican-3.7.1/classpelican_1_1readers_1_1RstReader.html
 
+_`.impl.design.github`: Design documents are also `rendered as HTML by GitHub <https://docs.github.com/en/repositories/working-with-files/using-files/working-with-non-code-files#rendering-differences-in-prose-documents>`__.
+
 _`.impl.manual`: The manual is written in RST using Sphinx_ extensions
 and custom manual extensions (see `.ext`_).
 
@@ -206,18 +209,18 @@ is translated into a ``c:function`` directive::
     .. c:function:: void LandFinish(Land land)
 
 
-_`.fmt.function-decl`: A paragraph consisting of a macro
-declaration on a single line formatted as code, for example::
+_`.fmt.macro-decl`: A paragraph consisting of a macro declaration on a
+single line formatted as code, for example::
 
     ``RING_FOR(node, ring, next)``
 
 is translated into a ``c:macro`` directive::
 
     .. c:macro:: RING_FOR(node, ring, next)
 
-Macros are identified by having names consisting of capital letters,
-numbers, and underscore, or appearing in the list of exceptions given
-by the ``MACROS`` global in designs.py.
+_`.fmt.macro`: Macros are identified by having names consisting of
+capital letters, numbers, and underscore, or appearing in the list of
+exceptions given by the ``MACROS`` global in designs.py.
 
 _`.fmt.type-def`: A paragraph consisting of a type definition on a
 single line formatted as code, for example::
@@ -323,14 +326,16 @@ Document History
 ----------------
 
 - 2018-09-18 GDR_ Created based on [RB_2013-05-09]_.
+- 2023-02-15 RB_ Updating for migration to GitHub.
 
 .. _GDR: https://www.ravenbrook.com/consultants/gdr/
+.. _RB: mailto:rb@ravenbrook.com
 
 
 Copyright and License
 ---------------------
 
-Copyright © 2018–2020 `Ravenbrook Limited <https://www.ravenbrook.com/>`_.
+Copyright © 2018–2023 `Ravenbrook Limited <https://www.ravenbrook.com/>`_.
 
 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions are

manual/Makefile

@@ -1,7 +1,11 @@
 # Makefile for Sphinx documentation
+#
 # $Id$
+#
+# Keep consistent with build.txt section on building the MPS manual.
 
 # You can set these variables from the command line.
+PYTHON        = python3
 SPHINXOPTS    =
 SPHINXBUILD   = tool/bin/sphinx-build
 PAPER         =
@@ -10,7 +14,7 @@ BUILDDIR      = .
 # Internal variables.
 PAPEROPT_a4     = -D latex_paper_size=a4
 PAPEROPT_letter = -D latex_paper_size=letter
-ALLSPHINXOPTS   = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source
+ALLSPHINXOPTS   = -T -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source
 # the i18n builder cannot share the environment and doctrees with the others
 I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source
 
@@ -154,12 +158,11 @@ doctest: $(SPHINXBUILD)
 	@echo "Testing of doctests in the sources finished, look at the " \
 	      "results in $(BUILDDIR)/doctest/output.txt."
 
-tools: tool/bin/sphinx-build
+.PHONY: tools
+tools: tool/bin/pip requirements.pip
+	tool/bin/pip install -r requirements.pip
 
-tool/bin/sphinx-build: tool/bin/pip
-	tool/bin/pip install 'sphinx < 5' # Layout breaks in Sphinx >= 5.  See <https://github.com/Ravenbrook/mps/issues/121>.
-	@echo "You can now use \`make SPHINXBUILD=tool/bin/sphinx-build html' etc."
-	@echo "Or add tool/bin to your path."
+tool/bin/sphinx-build: tools
 
 tool/bin/pip:
-	python3 -mvenv tool
+	$(PYTHON) -mvenv tool

manual/build.txt

@@ -9,7 +9,7 @@ Introduction
 ------------
 
 This document describes the various ways in which you can build the MPS,
-its libraries, and the tests and tools that come with it.
+its manual, its libraries, and the tests and tools that come with it.
 
 You may be building the MPS for a number of different purposes.
 
@@ -94,6 +94,32 @@ If you want to do anything beyond these simple cases, use the MPS build
 as described in the section "Building the MPS for development" below.
 
 
+Building the MPS manual
+-----------------------
+
+Builds of the MPS manual from the main MPS repo should be available at
+`<https://memory-pool-system.readthedocs.io/>`_.
+
+If that's not available, or if you have a variant of the MPS Kit, or
+are making modifications to the MPS itself, then you should build the
+manual for yourself.  This uses Sphinx
+`<https://www.sphinx-doc.org/>`_.
+
+On Unix-like platforms (including macOS), the Makefile in the manual
+directory can fetch and install a local copy of Sphinx and build the
+manual, like this::
+
+  cd manual
+  make html
+
+then open manual/html/index.html.
+
+On Windows platforms, follow the `Sphinx installation instructions
+<https://www.sphinx-doc.org/en/master/usage/installation.html>`_ for
+Windows, then invoke Sphinx as shown in the Makefile in the manual
+directory.
+
+
 Building the MPS for development
 --------------------------------
 

manual/requirements.pip

@@ -0,0 +1,4 @@
+# manual/requirements.txt -- MPS manual build requirements for pip
+
+# Layout breaks in Sphinx >= 5.  See <https://github.com/Ravenbrook/mps/issues/121>.
+sphinx >= 4, < 5

manual/source/extensions/mps/designs.py

@@ -1,11 +1,14 @@
 # designs.py -- Convert MPS design documents into sources for the MPS manual
-# $Id$
 #
-# This was originally done in the Makefile, but moved to Python in the hope
-# that readthedocs.org would be able to generate the manual.  However, they
-# (sensibly) don't run extensions, so we can't use them.  Processing the
-# designs here still seems like a good idea, though.
+# Copyright (c) 2013-2023 Ravenbrook Limited. See end of file for license.
+#
+# This code converts the MPS design documents (in plain
+# reStructuredText) for use in the manual (in Sphinx extended
+# reStructuredText) and so implements design.mps.doc.fmt.
 #
+# TODO: It's a shame that the regexps we convert from (near the start
+# of the file) are a log way from the strings we convert to (in
+# ``convert_file``).
 
 from __future__ import unicode_literals
 
@@ -43,22 +46,43 @@
 '''
 
 mode = re.compile(r'\.\. mode: .*\n')
+
+# design.mps.doc.metadata.tag
 prefix = re.compile(r'^:Tag: ([a-z][a-z.0-9-]*[a-z0-9])$', re.MULTILINE)
+
 rst_tag = re.compile(r'^:(?:Author|Date|Status|Revision|Copyright|Organization|Format|Index terms|Readership):.*?$\n', re.MULTILINE | re.IGNORECASE)
+
+# design.mps.doc.fmt.tag
 mps_tag = re.compile(r'_`\.([a-z][A-Za-z.0-9_-]*[A-Za-z0-9])`:')
+
+# design.mps.doc.fmt.ref
 mps_ref = re.compile(r'`(\.[a-z][A-Za-z.0-9_-]*[A-Za-z0-9])`_(?:        )?')
+
+# design.mps.doc.fmt.function-decl
 funcdef = re.compile(r'^``([^`]*\([^`]*\))``$', re.MULTILINE)
+
+# design.mps.doc.fmt.macro-decl
 macrodef = re.compile(r'^``((?:[A-Z][A-Z0-9_]+|{})(?:\([^`]*\))?)``$'
                       .format('|'.join(map(re.escape, MACROS.split()))), re.MULTILINE)
+# design.mps.doc.fmt.macro
 macro = re.compile(r'``([A-Z][A-Z0-9_]+)``(?:       )?')
+
+# design.mps.doc.fmt.type-def
 typedef = re.compile(r'^``typedef ([^`]*)``$', re.MULTILINE) 
+
+# design.mps.doc.fmt.function-ref
 func = re.compile(r'``([A-Za-z][A-Za-z0-9_]+\(\))``')
+
+# design.mps.doc.fmt.type-ref
 typename = re.compile(r'``({0}|[A-Z][A-Za-z0-9_]*'
                       r'(?:Class|Function|Method|Struct|Union)|'
                       r'mps_[a-z_]+_[stu])``(?:      )?'
                       .format('|'.join(map(re.escape, TYPES.split()))))
+
+# Tags referencing other design documents
 design_ref = re.compile(r'^( *\.\. _design\.mps\.(?:[^:\n]+): (?:[^#:\n]+))$', re.MULTILINE)
 design_frag_ref = re.compile(r'^( *\.\. _design\.mps\.([^:\n]+)\.([^:\n]+): (?:[^#:\n]+))#(.+)$', re.MULTILINE)
+
 relative_link = re.compile(r'^( *\.\. _[\w\.]+: +\.\./)', re.MULTILINE)
 history = re.compile(r'^Document History\n.*',
                      re.MULTILINE | re.IGNORECASE | re.DOTALL)
@@ -69,7 +93,9 @@
 def secnum_sub(m):
     return m.group(1) + '\n' + m.group(3) * len(m.group(1))
 
-# Convert Ravenbrook style citations into MPS Manual style citations.
+# Convert Ravenbrook style citations into MPS Manual style citations
+# (design.mps.doc.fmt.citation)
+#
 # Example citations transformation, from:
 #     .. [THVV_1995] "Structure Marking"; Tom Van Vleck; 1995;
 #        <http://www.multicians.org/thvv/marking.html>.
@@ -157,18 +183,66 @@ def newer(src, target):
 logger = logging.getLogger(__name__)
 
 # Mini-make
+#
+# This uses app.srcdir (the directory of the manual sources) to find
+# the root of the MPS tree (two directories above) and so find the
+# design documents to convert.
+
 def convert_updated(app):
     logger.info(bold('converting MPS design documents'))
-    for design in glob.iglob('../design/*.txt'):
+    for design in glob.iglob(os.path.join(app.srcdir, '../../design/*.txt')):
         name = os.path.splitext(os.path.basename(design))[0]
         if name == 'index': continue
-        converted = 'source/design/%s.rst' % name
+        converted = os.path.join(app.srcdir, 'design/%s.rst' % name)
         if newer(design, converted):
             logger.info('converting design %s' % name)
             convert_file(name, design, converted)
-    diagrams = chain(*[glob.iglob('../design/*.' + ext)
+    diagrams = chain(*[glob.iglob(os.path.join(app.srcdir, '../../design/*.' + ext))
                        for ext in 'png svg'.split()])
     for diagram in diagrams:
-        target = os.path.join('source/design/', os.path.basename(diagram))
+        target = os.path.join(app.srcdir, 'design/', os.path.basename(diagram))
         if newer(diagram, target):
             shutil.copyfile(diagram, target)
+
+
+# A. REFERENCES
+#
+# [GDR-2018-09-18] "Documentation"; Gareth Rees; 2018-09-18; design.mps.doc.
+#
+#
+# B. DOCUMENT HISTORY
+#
+# 2013-06-01 RB Created.
+# 2023-02-15 RB Updated with copyright, licence, and document history.
+#
+#
+# C. COPYRIGHT AND LICENSE
+#
+# Copyright (C) 2023 Ravenbrook Limited <https://www.ravenbrook.com/>.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met:
+#
+# 1. Redistributions of source code must retain the above copyright
+#    notice, this list of conditions and the following disclaimer.
+#
+# 2. Redistributions in binary form must reproduce the above copyright
+#    notice, this list of conditions and the following disclaimer in the
+#    documentation and/or other materials provided with the
+#    distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
+# IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+#
+#
+# $Id$

manual/source/guide/overview.rst

@@ -8,6 +8,9 @@
 Overview of the Memory Pool System
 ==================================
 
+.. IMPORTANT: If you change the paragraph below, also change
+   readme.txt
+
 The Memory Pool System is a very general, adaptable, flexible,
 reliable, and efficient memory management system. It permits the
 flexible combination of memory management techniques, supporting