Meta-ticket: Sage docbuild

[nthiery]

Docbuild tickets:

• Split sage_setup.docbuild out to a separate distribution sage_docbuild #30010 Split sage_setup.docbuild out to a separate distribution (distutils package)
• sage --docbuild: Add options to list all documents #31353 sage --docbuild: Add options to list all documents
• docbuild: Japanese characters creeping into other parts of Sage documentation #31936 docbuild: Japanese characters creeping into other parts of Sage documentation
• docbuild: switch from optparse to argparse #31366 docbuild: switch from optparse to argparse
• Reimplement parallel docbuild using make #31948 Reimplement parallel docbuild using make
• pip-installable packages sagemath-doc-src, sagemath-doc-inventory, sagemath-doc-html, sagemath-doc-pdf #29868 pip-installable packages sage-doc-html, sage-doc-pdf
• Parallel docbuild, cythonization: Use GNU make's POSIX jobserver protocol #30369 Parallel docbuild, cythonization: Use GNU make's POSIX jobserver protocol
• sage -docbuild: error building docs of single file with relative imports #30475 sage -docbuild: error building docs of single file with relative imports
• expose building documentation for user modules not in the Sage library #6389 expose building documentation for user modules not in the Sage library
• Missing PDF icons in the Sage documentation website #30418 Missing PDF icons in the Sage documentation website
• Shrink Documentation #28376 Shrink Documentation
• GH Actions workflow that builds documentation on each push #31415 GH Actions workflow that deploys built documentation
• pip-installable packages sagemath-doc-src, sagemath-doc-inventory, sagemath-doc-html, sagemath-doc-pdf #29868 pip-installable packages sagemath-doc-src, sagemath-doc-inventory, sagemath-doc-html, sagemath-doc-pdf
• Versioned Sage documentation documentation#24 (Versioned Sage documentation)

Sphinx tickets:

• Sphinx/docutils update for Sage 9.4 #31696 Sphinx/docutils update for Sage 9.4
• Use sphinx.util.inspect.Signature not sphinx.ext.autodoc.inspector.formatargspec #27578 Use sphinx.util.inspect.Signature not sphinx.ext.autodoc.inspector.formatargspec
• link sphinx docs of Sage components in reference manual #27164 link sphinx docs of Sage components in reference manual
• add intersphinx mapping for SciPy #29231 add intersphinx mapping for SciPy
• Use sphinx.ext.extlinks to add Sphinx roles for links to external package docs #27495 Use sphinx.ext.extlinks to add Sphinx roles for links to external package docs
• Replace sage autodoc extension by built-in one #30893 Replace sage autodoc extension by built-in one
• Add typing info to documentation using sphinx_autodoc_typehints #30894 Add typing info to documentation using sphinx_autodoc_typehints

See also:

• Meta-ticket: Improve live documentation in the Jupyter notebook #20893 Meta-ticket: Improve live documentation in the Jupyter notebook

---

Overview of Sphinx custom code in Sage's documentation build system

The goal of this ticket is to write an overview of the various bits and pieces of custom Sphinx code in Sage, with the long term goal to see which pieces could be:

• thrown away, using features that are now in Sphinx
• generalized and submitted upstream
• cleaned up

Here is a list of files with their relative roles:

src/sage/docs

• __init__.py: empty file

• conf.py: standard Sphinx configuration file.

src/sage_docbuild (was src/sage_setup/docbuild)

• __init__.py: top-level docbuilder, contains classes for the various documents (e.g. reference manual).

• __main__.py: stub entry point for sage --docbuild, calls main() from __init__.py

• build_options.py: Parse the option for building.

• sphinxbuild.py: Sage's version of the sphinx-build script

  mainly deals with logging and error reporting.

src/sage_docbuild/ext (was src/sage_setup/docbuild/ext)

• inventory_builder.py:

  A customized HTML builder which only generates intersphinx "object.inv"
  inventory files and pickle files. The documentation files are not written.
  This is supposed to be used with multidocs below.

• multidocs.py:

  The goal of this extension is to manage a multi documentation in Sphinx.
  To be able to compile Sage's huge documentation in parallel, the
  documentation is cut into a bunch of independent documentations called
  "subdocs", which are compiled separately. There is a master document which
  points to all the subdocs. The intersphinx extension ensures that the
  cross-link between the subdocs are correctly resolved. However some work
  is needed to build a global index. This is the goal of multidocs.

  More precisely this extension ensures the correct merging of

    1. the todo list if this extension is activated;
    2. the python indexes;
    3. the list of python modules;
    4. the javascript index;
    5. the citations.
  

• sage_autodoc.py

  This is a patched version of sphinx.ext.autodoc, which started to diverge
  before 2011. As far as I (Florent) understand, the role here is to

    1. allows to correctly fetch the Cython doc and argspec
    2. get correctly the argspec of callable (from the `_sage_argspec_`
       attribute) #9976
    3. deals correctly with LazyImport #17455
    4. In particular document CachedFunction as a function and not a class
       instance #9976.
    5. correctly handles classe aliases and nested classed #7448 #5986
  

  This file probably need to be completely reworked since Sphinx seems to
  have now various plugin and Mixin.

src/sage/misc

• sphinxify.py

  script which calls Sphinx to format a single docstring for help from the
  command line or the notebook. Has its own configuration.

CC: @egourgoulhon @jhpalmieri @dimpase @kwankyu @haraldschilly

Component: documentation

Issue created by migration from https://trac.sagemath.org/ticket/20080

[kiwifb]

comment:6

I would like to mention sphinxify.py which is currently part of sagenb but should be moved to inside sage proper ASAP. sphinxify is called from sage/misc/sagedoc.py both at doc building time and run time.

[kcrisman]

comment:10

I would like to mention sphinxify.py which is currently part of sagenb but should be moved to inside sage proper ASAP. sphinxify is called from sage/misc/sagedoc.py both at doc building time and run time.

I'm open to PRs, as this seems pretty reasonable - docbuild shouldn't depend on sagenb, though we would still want the doc to work from within it.

[embray]

comment:12

Could someone clarify what

This is a slightly hacked-up version of the Sphinx-multidoc plugin

means?

I can't find any reference to a Sphinx multidoc plugin outside of Sage.

[jdemeyer]

comment:13

Fixed :-)

[embray]

comment:15

I'm having "fun" finally taking the time to fully understand the doc builds system... :)

[dimpase]

comment:16

it would be a good idea to revise this, or perhaps just start from scratch. It's not clear to me whether most/all of this can be achieved with vanilla Sphinx 3 or 4.

[mkoeppe]

comment:17

Related: #30010 - Split sage_setup.docbuild out to a separate distribution (distutils package)

[dimpase]

comment:18

One of sphinx devs,
Takeshi KOMIYA, is going to do a comparison with "native" sphinx for the reference manual - sphinx-doc/sphinx#7891

[nthiery]

comment:19

Cool!

[mkoeppe]

comment:23

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.