Build documentation in $SAGE_SHARE/doc/sage

[jdemeyer]

Depends on #19127

CC: @hivert @mezzarobba

Component: build

Author: Jeroen Demeyer

Branch: 96685ab

Reviewer: François Bissey

Issue created by migration from https://trac.sagemath.org/ticket/19963

[jdemeyer]

Branch: u/jdemeyer/build_documentation_in__sage_local_share_doc_sage

[jdemeyer]

Commit: 2c56897

[jdemeyer]

New commits:

7c98158	Docbuilder cleanup
537d9d2	Minor fixes
1cebbc9	Fix citation dir with custom SAGE_DOC_OUTPUT
2c56897	Build documentation in $SAGE_SHARE/doc/sage

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

dfe2c06

Minor fixes to top-level build system

[sagetrac-git]

Changed commit from 2c56897 to dfe2c06

[sagetrac-git]

Changed commit from dfe2c06 to cb2b63f

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

cb2b63f

Remove old src/doc/ouput and .pyc files

[mezzarobba]

comment:6

This will break the “Help” links in the (classic) notebook, won't it? More generally, though I entirely agree with the goal of this ticket, I'm not sure people like the change. Wouldn't it be possible to keep a compatibility symlink or something? Also, several places in the documentation (starting with README.txt) explicitly refer the reader to src/doc/output.

[kcrisman]

comment:7

Yeah, I'm not sure exactly what the necessity of this is. At the very least it would be something where "deprecation" would be useful?

[jdemeyer]

comment:8

Replying to @kcrisman:

I'm not sure exactly what the necessity of this is.

Everything which is compiled/built/installed in the Sage distribution ends up in $SAGE_LOCAL... except the documentation. So it's logical to put the built documentation in $SAGE_LOCAL too.

At the very least it would be something where "deprecation" would be useful?

Well, you cannot really deprecate filesystem paths. But a symbolic link can be done indeed.

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

a601a8c	Add symbolic link SAGE_DOC/output -> SAGE_DOC_OUTPUT
de10a5e	Use new documentation paths in documentation

[sagetrac-git]

Changed commit from cb2b63f to de10a5e

[jdemeyer]

comment:11

Replying to @mezzarobba:

This will break the “Help” links in the (classic) notebook, won't it?

Fixed in sagemath/sagenb#363

[kiwifb]

comment:12

I am unfortunately taking this train after it left the station. I guess the code re-organisation had to happen in #19127 and all. But what a choice of variable names. Seriously SAGE_DOC_OUTPUT? Yes I can see where it came from. But seriously I expect my doc to be found under SAGE_DOC not SAGE_DOC_OUTPUT, that's the natural name to use. The other one could have been SAGE_DOC_SRC, and if it isn't used after install I am not sure I would bother putting it in env.py.

[jdemeyer]

comment:13

Replying to @kiwifb:

I am unfortunately taking this train after it left the station. I guess the code re-organisation had to happen in #19127 and all. But what a choice of variable names. Seriously SAGE_DOC_OUTPUT? Yes I can see where it came from. But seriously I expect my doc to be found under SAGE_DOC not SAGE_DOC_OUTPUT, that's the natural name to use. The other one could have been SAGE_DOC_SRC, and if it isn't used after install I am not sure I would bother putting it in env.py.

I will change the variable names on this ticket if somebody commits to reviewing this ticket then.

[kiwifb]

comment:14

I am going through the changes in #19127 first. You made it incredibly difficult to build the documentation before having installed sage at all. Unless I was lucky before (possible but unlikely considering the amount of stuff that can go wrong) that's a regression as far as I am concerned. But it is too late for this in that ticket or this ticket.

The code here looks ok to me, it is mostly house keeping.

The html and the pdf documentations are moved to a new home and only the rst file are left behind. The only mention left of SAGE_DOC in the current form is in sage/misc/sagedoc.py, possibly too hard for this ticket but if it is only for building at least some parts of sagedoc.py could/should be moved to sage_setup/docbuild.
There is also some mention of SAGE_DOC and SAGE_DOC/output in sage/misc/latex_macros.py. Only in documentation and comment blocks. At least two of these should be replaced by SAGE_DOC_OUTPUT, the first instance is to quote a path in sage's source tree. Ultimately this should be shipped too in my opinion but at this stage is the only instance of SAGE_DOC that is not superfluous once that sage is installed - outside of sage_setup/docbuild.

[kiwifb]

comment:15

In sage_setup/docbuild/__init__.py we currently have at line 101 and after

        if 'output/html' in output_dir:
            logger.warning("Build finished. The built documents can be found in %s",
                           output_dir)
        elif 'output/pdf' not in output_dir:
            logger.info("Build finished. The built documents can be found in %s",
                           output_dir)
        else:
            logger.info("LaTeX file written to %s; now making PDF.",
                           output_dir)

Will we ever see a message "Build finished. The built documents can be found in..." again?

[jhpalmieri]

comment:16

You made it incredibly difficult to build the documentation before having installed sage at all.

When was this ever possible? Certainly Sphinx needs to be installed, and docbuilding also reads parts (or all?) of the Sage library.

[kiwifb]

comment:17

Replying to @jhpalmieri:

You made it incredibly difficult to build the documentation before having installed sage at all.

When was this ever possible? Certainly Sphinx needs to be installed, and docbuilding also reads parts (or all?) of the Sage library.

Of course sphinx was installed. I meant the sage library itself. And yes it was possible, may still be possible with a bit of care. What happens technically is that the docbuilding itself only needs to go through sage's sources, and if you don't call anything from cythonized modules during the building process itself, you don't need anything but the sources. You started to import cythonized stuff (cachefunc for starters) and things just broke. It look like you can do stuff from build/lib but attention to details is required.

[kiwifb]

comment:18

Sorry John, I thought that was from Jeroen.

[jdemeyer]

comment:19

Replying to @kiwifb:

And yes it was possible

Doesn't autodoc import the modules that it builds the documentation of, to determine __doc__? If so, there is no way to build the Sage documentation without building Sage first.

[jdemeyer]

comment:20

Replying to @kiwifb:

You started to import cythonized stuff (cachefunc for starters)

No, the Sage docbuilder has always used cachefunc, that is nothing new.

[kiwifb]

comment:22

Replying to @jdemeyer:

Replying to @kiwifb:

You started to import cythonized stuff (cachefunc for starters)

No, the Sage docbuilder has always used cachefunc, that is nothing new.

Let's not get into this any more and finish this ticket. I may be looking for something stupid.

[sagetrac-git]

Changed commit from de10a5e to 96685ab

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

96685ab

Fix a few more paths

[kiwifb]

Reviewer: François Bissey

[kiwifb]

comment:25

I am happy with that. Anything further will be for a follow up ticket. And once the PR to sagenb is included in a release we should remove the creation of links to the old location. sagenb is really the only thing that currently depends on it.

[vbraun]

Changed branch from u/jdemeyer/build_documentation_in__sage_local_share_doc_sage to 96685ab

[jhpalmieri]

comment:27

For a followup ticket: should we reinstate the line rm -rf output in src/doc/Makefile? Otherwise the old output will remain and possibly confuse people. Or should the old output be removed somewhere else?

[jhpalmieri]

Changed commit from 96685ab to none

[kiwifb]

comment:28

Replying to @jhpalmieri:

For a followup ticket: should we reinstate the line rm -rf output in src/doc/Makefile? Otherwise the old output will remain and possibly confuse people. Or should the old output be removed somewhere else?

That would be a good idea I would say. Not quite on the top of my laundry list. But if we want to discuss those matters, it shouldn't be on this ticket.

[jdemeyer]

comment:29

Old docs are deleted, see src/sage_setup/docbuild/__main__.py:

    if os.path.exists(old_doc):
        from shutil import rmtree
        rmtree(old_doc)