Skip to content

Overhaul Building the Documentation section for clarity #1038

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Mar 6, 2023
Merged

Overhaul Building the Documentation section for clarity #1038

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0a0b340
Overhaul Building the Documentation section for clarity
CAM-Gerlach Jan 25, 2023
5f4ccf2
Apply suggestions and further clarifications from Hugo
CAM-Gerlach Jan 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 79 additions & 49 deletions documentation/start-documenting.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,79 +64,109 @@ Building the documentation

.. highlight:: bash

The toolset used to build the docs is written in Python and is called Sphinx_.
Sphinx is maintained separately and is not included in this tree. Also needed
are blurb_, a tool to create :file:`Misc/NEWS` on demand; and
python-docs-theme_, the Sphinx theme for the Python documentation.
To build the documentation, follow the steps in one of the sections below.
You can view the documentation after building the HTML
by opening the file :file:`Doc/build/html/index.html` in a web browser.

To build the documentation, follow the instructions from one of the sections
below. You can view the documentation after building the HTML by pointing
a browser at the file :file:`Doc/build/html/index.html`.
.. note::

You are expected to have installed the latest stable version of
Sphinx_ and blurb_ on your system or in a virtualenv_ (which can be
created using ``make venv``), so that the Makefile can find the
``sphinx-build`` command. You can also specify the location of
``sphinx-build`` with the ``SPHINXBUILD`` :command:`make` variable.
The following instructions all assume your current working dir is
the ``Doc`` subdirectory in your :ref:`CPython repository clone <checkout>`.
Make sure to switch to it with ``cd Doc`` if necessary.


.. _building-using-make:
.. _doc-create-venv:

Create a virtual environment
----------------------------

Using make / make.bat
---------------------
.. _doc-create-venv-unix:

**On Unix**, run the following from the root of your :ref:`repository clone
<checkout>` to build the output as HTML::
**On Unix platforms** that support :program:`make`
(including Linux, macOS and BSD),
you can create a new :mod:`venv` with the required dependencies using::

cd Doc
make venv
make html

or alternatively ``make -C Doc/ venv html``. ``htmlview`` can be used
instead of ``html`` to conveniently open the docs in a browser once the
build completes.
Building the docs with :program:`make` will automatically use this environment
without you having to activate it.

.. _doc-create-venv-windows:

**On Windows**, or if not using :program:`make`,
`create a new virtual environment <venv-create_>`__ manually.
Always be sure to `activate this environment <venv-activate_>`__
before building the documentation.


.. _building-using-make:
.. _using-make-make-bat:
.. _doc-build-make:

Build using make / make.bat
---------------------------

You can also use ``make help`` to see a list of targets supported by
:command:`make`. Note that ``make check`` is automatically run when
you submit a :ref:`pull request <pullrequest>`, so you should make
sure that it runs without errors.
A Unix ``Makefile`` is provided, :cpy-file:`Doc/Makefile`,
along with a :cpy-file:`Doc/make.bat` batch file for Windows
that attempts to emulate it as closely as practical.

**On Windows**, the :cpy-file:`Doc/make.bat` batchfile tries to emulate
:command:`make` as closely as possible, but the venv target is not implemented,
so you will probably want to make sure you are working in a virtual environment
before proceeding, otherwise all dependencies will be automatically installed
on your system.
.. important::

When ready, run the following from the root of your :ref:`repository clone
<checkout>` to build the output as HTML::
The Windows ``make.bat`` batch file lacks a ``make venv`` target.
Instead, it automatically installs any missing dependencies
into the currently activated environment (or the base Python, if none).
Make sure the environment you :ref:`created above <doc-create-venv-windows>`
is `activated <venv-activate_>`__ before running ``make.bat``.

To build the docs as HTML, run::

cd Doc
make html

You can also use ``make help`` to see a list of targets supported by
:cpy-file:`Doc/make.bat`.
.. tip:: Substitute ``htmlview`` for ``html`` to open the docs in a web browser
once the build completes.

To check the docs for common errors with `Sphinx Lint`_
(which is run on all :ref:`pull requests <pullrequest>`), use::

make check

To list other supported :program:`make` targets, run::

make help

See :cpy-file:`Doc/README.rst` for more information.


.. _using-sphinx-build:
.. _doc-build-sphinx:

Build using Sphinx directly
---------------------------

Advanced users may want to invoke Sphinx directly,
to pass specialized options or to handle specific use cases.

See also :cpy-file:`Doc/README.rst` for more information.
Make sure the environment you :ref:`created above <doc-create-venv-windows>`
is `activated <venv-activate_>`__.
Then, install the documentation requirements, :cpy-file:`Doc/requirements.txt`.
Using pip::

Using sphinx-build
------------------
python -m pip install --upgrade -r requirements.txt

Sometimes we directly want to execute the sphinx-build tool instead of through
``make`` (although the latter is still the preferred way). In this case, you can
use the following command line from the ``Doc`` directory (make sure to install
Sphinx_, blurb_ and python-docs-theme_ packages from PyPI)::
Finally, directly invoke Sphinx with::

sphinx-build -b<builder> . build/<builder>
python -m sphinx -b html . build/html
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also -j auto to 'build in parallel with N processes where possible (special value "auto" will set N to cpu-count)'?

Suggested change
python -m sphinx -b html . build/html
python -m sphinx -b html . build/html -j auto

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, perhaps, but since this is primarily intended to provide a minimal working invocation specifically for advanced users who want to pass their own custom options anyway, I'm not sure whether it makes sense to add here, considering it is entirely optional, can easily be added if desired, doesn't appear to do anything on Windows (the platform most likely to use it), and may not make sense for many use cases for which the direct invocation would be used, e.g. rebuilding only a single file or running on CI.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shame it doesn't work on Windows.

So it will either do nothing or make things much quicker: 62s -> 41s on my machine, which I think is a good win without much complexity. Especially if this bit is intended for advanced users.

And some Linux/Mac users who could use make will prefer to call things directly.

Anyway, I don't mind too much, take your pick here too!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm kinda torn here...it's nice that it speeds things up for some people, but adding extra optional, non-self-documenting arguments to the baseline invocation causes me a bit of concern over complicating things for users, and them retaining it in cases where it is not helpful (e.g. rebuilding only one file). Though, on the other hand, under the same assumption that users care about the arguments they are passing, they could just as easily be aware of what -j auto does, or be looking up the arguments anyway.

Shame it doesn't work on Windows.

My understanding from what Adam told me is its due to lack of fork support, which makes spinning up new processes very expensive and rarely worth the trouble for the types of workloads Sphinx is typically run on.

And some Linux/Mac users who could use make will prefer to call things directly.

FWIW, I have a working make installed through Rtools on Windows (and also exclusively use Git Bash), and use it myself to build the Spyder Docs I maintain (since we have a fairly complex workflow for multi-version support), but don't use it for the CPython-related projects that use it to build docs, since it assumes an in-tree venv (whereas I'm using conda environments), I need precise control over both the Python invocation and the arguments passed to Sphinx, and I prefer a consistent invocation and behavior, with no extra magic, across the many docs projects I work on.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does python -m sphinx not accept --parallel, as a synonym for -j? If not (and AFAICT it truly, in fact, does not), that's probably something that should be fixed (in sphinx), and would at least help to address the "non-self-documenting" aspect.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I've submitted sphinx-doc/sphinx#11169 to do just that.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-j is usually a shortcut for --jobs. From make --help:

  -j [N], --jobs[=N]          Allow N jobs at once; infinite jobs with no arg.


where ``<builder>`` is one of html, text, latex, or htmlhelp (for explanations
see the make targets above).
To use a different `Sphinx builder`_,
replace ``html`` above with the desired builder ``name``.


.. _docutils: https://docutils.sourceforge.io/
.. _python-docs-theme: https://pypi.org/project/python-docs-theme/
.. _Sphinx: https://www.sphinx-doc.org/
.. _virtualenv: https://virtualenv.pypa.io/
.. _blurb: https://pypi.org/project/blurb/
.. _Sphinx builder: https://www.sphinx-doc.org/en/master/usage/builders/index.html
.. _Sphinx Lint: https://github.com/sphinx-contrib/sphinx-lint
.. _venv-activate: https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/#activating-a-virtual-environment
.. _venv-create: https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/#creating-a-virtual-environment


Style Guide
Expand Down