Update Sphinx to 1.4.1

[a-andre]

Changes: http://sphinx-doc.org/changes.html

Tarballs:

• https://pypi.python.org/packages/source/i/imagesize/imagesize-0.7.0.tar.gz
• https://pypi.python.org/packages/source/s/snowballstemmer/snowballstemmer-1.2.1.tar.gz
• https://pypi.python.org/packages/6e/65/3b15c248e4078278a7c9d45ecac2b87ae56f03dfb574e6ee31edea360fdd/alabaster-0.7.7.tar.gz
• https://pypi.python.org/packages/source/S/Sphinx/Sphinx-1.4.1.tar.gz

Depends on #20496
Depends on #20498

CC: @kiwifb

Component: packages: standard

Keywords: days77

Author: André Apitzsch, Jeroen Demeyer

Branch/Commit: bdc3f0b

Reviewer: Steven Trogdon, John Palmieri

Issue created by migration from https://trac.sagemath.org/ticket/18497

[a-andre]

Commit: c913a28

[a-andre]

Branch: u/aapitzsch/ticket/18497

[a-andre]

comment:1

The given branch misses some dependencies (babel, snowballstemmer). Installing them via pip makes Sphinx build. But documentation failed to build.

[sagetrac-git]

Changed commit from c913a28 to 832afb7

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

ede0305	update sphinx package to 1.3.1
7d29133	fix some deprecation warnings
557a33c	rebase sage_autodoc changes on sphinx.ext.autodoc version 1.3.1
bdf70fe	fix orphan setting
832afb7	fix warning "more than one target found for cross-reference u'groebner_basis'"

[a-andre]

comment:3

Any ideas how to fix

Traceback (most recent call last):
  File "/opt/sage/src/doc/common/builder.py", line 1626, in <module>
    getattr(get_builder(name), type)()
  File "/opt/sage/src/doc/common/builder.py", line 292, in _wrapper
    getattr(get_builder(document), 'inventory')(*args, **kwds)
  File "/opt/sage/src/doc/common/builder.py", line 503, in _wrapper
    x.get(99999)
  File "/opt/sage/local/lib/python/multiprocessing/pool.py", line 558, in get
    raise self._value
OSError: [categorie] /opt/sage/src/doc/en/reference/categories/sage/categories/category.rst:370: WARNING: don't know which module to import for autodocumenting u'_super_categories' (try placing a "module" or "currentmodule" directive in the document, or giving an explicit module name)

without changing

.. automethod:: _super_categories

to

.. automethod:: Category._super_categories

because there are a lot of these warnings.

[strogdon]

comment:5

While building sphinx I get:

Installed /64bitdev/storage/sage-git_develop/sage/local/lib/python2.7/site-packages/Sphinx-1.3.1-py2.7.egg
Processing dependencies for Sphinx==1.3.1
Searching for snowballstemmer>=1.1
Reading https://pypi.python.org/simple/snowballstemmer/
Download error on https://pypi.python.org/simple/snowballstemmer/: [Errno 110] Connection timed out -- Some packages may not be found!
Couldn't find index page for 'snowballstemmer' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
Download error on https://pypi.python.org/simple/: [Errno 110] Connection timed out -- Some packages may not be found!
No local packages or download links found for snowballstemmer>=1.1
error: Could not find suitable distribution for Requirement.parse('snowballstemmer>=1.1')
Error building Sphinx: 'Error installing Sphinx'

wget works just fine in downloading snowballstemmer from pypi.python.org/simple/snowballstemmer

[strogdon]

comment:6

The inability to download snowballstemmer>=1.1 seems to be because of

export http_proxy=http://192.0.2.0:5187/

in build/bin/sage-spkg. This was supposedly fixed in #14243. Am I the only one getting this?

[jdemeyer]

comment:7

Replying to @strogdon:

The inability to download snowballstemmer>=1.1 seems to be because of

export http_proxy=http://192.0.2.0:5187/

in build/bin/sage-spkg. This was supposedly fixed in #14243. Am I the only one getting this?

I don't see how #14243 is related. The proxy is obviously intentional, read the comment.

[strogdon]

comment:8

Replying to @jdemeyer:

I don't see how #14243 is related. The proxy is obviously intentional, read the comment.

You're right about #14243 not being related. I was a bit hasty. And I realize the proxy is intentional. But what's the work-around here to allow Sphinx to install correctly? If I'm not mistaken, the proxy is blocking snowballstemmer download. But maybe there is something else.

[jdemeyer]

comment:9

Replying to @strogdon:

But what's the work-around here to allow Sphinx to install correctly?

Add a snowballstemmer package.

[jhpalmieri]

comment:10

This works for me, but not perfectly. In addition to the changes here, I used sage --pip install to install snowballstemmer and alabaster. I also had to change exclude_trees to exclude_patterns wherever it occurred in doc/common: exclude_trees was deprecated before and has now been removed. (This was in src/doc/common/conf.py, src/doc/en/reference/conf.py, src/doc/en/reference/conf_sub.py.)

I ran into the issue mentioned in comment:3, so I made those references more explicit (changed _blah to Category._blah, etc.). Then the file sage/lfunctions/zero_sums.pyx was giving me errors I didn't understand, for example

OSError: [lfunction] docstring of sage.lfunctions.zero_sums.LFunctionZeroSum_EllipticCurve:0: ERROR: Unknown interpreted text role "class".

So I removed it from the reference manual. I'll keep looking at that file, though.

Then in the second pass through the reference manual, the build stopped with complaints about missing references or citations. Maybe Sphinx is pickier now, and wants the reference before any citations to it, so I moved some of them to the top-level docstring in the file, and that helped. There is a few cross-file references, and I don't know what to do about that one except to run Sphinx again and hope it resolves the reference. The build completed this way.

By the way, Sphinx 1.3.2 (a bug-fix release) is supposed to be released in a few days.

[strogdon]

comment:11

I have tried the same (I missed the exclude_trees deprecation) and I too see in the build log

[lfunction] docstring of sage.lfunctions.zero_sums.LFunctionZeroSum_EllipticCurve:0: ERROR: Unknown interpreted text role "class".
[lfunction] docstring of sage.lfunctions.zero_sums.LFunctionZeroSum_EllipticCurve.analytic_rank_upper_bound:83: ERROR: Unknown interpreted text role "func".
[lfunction] docstring of sage.lfunctions.zero_sums.LFunctionZeroSum_EllipticCurve.analytic_rank_upper_bound:83: ERROR: Unknown interpreted text role "meth".
[lfunction] docstring of sage.lfunctions.zero_sums.LFunctionZeroSum_EllipticCurve.analytic_rank_upper_bound:83: ERROR: Unknown interpreted text role "func".
[lfunction] docstring of sage.lfunctions.zero_sums.LFunctionZeroSum_abstract:0: ERROR: Unknown interpreted text role "class".
[lfunction] docstring of sage.lfunctions.zero_sums.LFunctionZeroSum_abstract.cnlist:19: ERROR: Unknown interpreted text role "meth".
[lfunction] docstring of sage.lfunctions.zero_sums.LFunctionZeroSum_abstract.completed_logarithmic_derivative:39: ERROR: Unknown interpreted text role "meth".
[lfunction] docstring of sage.lfunctions.zero_sums.LFunctionZeroSum_abstract.zerosum:66: ERROR: Unknown interpreted text role "meth".

And I have been searching high and low to figure out the issue. Also, if alabaster is not needed then sphinx will have to be patched to prevent it from being imported. Something like:

--- Sphinx-1.3.1/sphinx/application.py	2015-09-11 22:55:30.000000000 -0500
+++ Sphinx-1.3.1/sphinx/application.py	2015-09-15 17:56:42.000000000 -0500
@@ -135,8 +135,8 @@
         # extension loading support for alabaster theme
         # self.config.html_theme is not set from conf.py at here
         # for now, sphinx always load a 'alabaster' extension.
-        if 'alabaster' not in self.config.extensions:
-            self.config.extensions.append('alabaster')
+        #if 'alabaster' not in self.config.extensions:
+        #    self.config.extensions.append('alabaster')
 
         # load all user-given extension modules
         for extension in self.config.extensions:

[jhpalmieri]

Changed branch from u/aapitzsch/ticket/18497 to u/jhpalmieri/ticket/18497

[jhpalmieri]

Changed commit from 832afb7 to 717f4e8

[jhpalmieri]

comment:13

I don't know why, but this change fixes the problem with lfunctions/zero_sum.pyx:

diff --git a/src/sage/lfunctions/zero_sums.pyx b/src/sage/lfunctions/zero_sums.pyx
index 6c4535c..ff1dd86 100644
--- a/src/sage/lfunctions/zero_sums.pyx
+++ b/src/sage/lfunctions/zero_sums.pyx
@@ -1341,7 +1341,6 @@ cdef class LFunctionZeroSum_EllipticCurve(LFunctionZeroSum_abstract):
 
         return small_primes, modulus, residue_chunks
 
-    @parallel(ncpus=NCPUS)
     def _sum_over_residues(self, residue_sum_data):
         r"""
         Return the p-power sum over residues in a residue chunk

I have created a branch for testing purposes which does this and the other changes I outlined above. It is not at all ready for review. Issues:

• removing this "parallel" decorator is probably not a good idea.
• snowballstemmer needs be added as a new package.
• Maybe alabaster, too? I don't see why not.
• The issue with missing references needs to be resolved.

---

New commits:

ad08dd1	Merge branch 'develop' into t/18497/ticket/18497
717f4e8	trac 18497: in docbuilding, use exclude_patterns instead of exclude_trees.

[strogdon]

comment:14

When I first started making changes to deal with the issue mentioned in comment:#3 I only had to make changes to these files

sage/categories/category.py
sage/categories/category_with_axiom.py
sage/sets/cartesian_product.py
sage/combinat/permutation.py

I thought this odd and suspected that something else was the reason requiring the changes.

[jhpalmieri]

comment:15

I wouldn't be surprised if something else was causing those problems. I didn't do a careful case-by-case test to see which changes were necessary.

[jdemeyer]

comment:16

The huge change to sage_autodoc.py really scares me a lot. I already have several tickets concerning the docbuilding sitting in needs_review and this ticket here will almost certainly cause a lot of conflicts.

I would much prefer to stick with the current version of Sphinx (unless you have a good reason to upgrade) and first clean up the Sphinx mess.

[jdemeyer]

comment:17

See also OpenDreamKit/OpenDreamKit#87

[jhpalmieri]

comment:18

Sphinx 1.3.2 has now been released.

[sagetrac-git]

Changed commit from e26d349 to 7c2d9d5

[jdemeyer]

comment:68

Replying to @strogdon:

Also, if alabaster is not needed then sphinx will have to be patched to prevent it from being imported.

I fixed this problem by simply installing alabaster. It's a small package, so it's not worth the trouble to patch Sphinx for it. Also, while Sage doesn't use alabaster, maybe other packages do use it.

[strogdon]

comment:69

Replying to @jhpalmieri:

Is this the formula on line 710 of multi_modular.pyx? That looks like it has extra backticks or dollar signs: the formula should have one or the other, not both.

Yes, that's it. And there is a similar problem in src/sage/symbolic/expression.pyx around line 8830.

 There are two square roots of `$(x + 1)^2$`

It appears this version of Sphinx is exposing some sphinx syntax issues in the documentation that now appear when generating the latex files for building the pdf-docs. The older sphinx -> latex must be a bit more forgiving.

[sagetrac-git]

Changed commit from 7c2d9d5 to f1ae5e6

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

f1ae5e6

Fix doctest failures with Sphinx 1.4.1

[jdemeyer]

comment:71

Replying to @strogdon:

Yes, that's it. And there is a similar problem in src/sage/symbolic/expression.pyx around line 8830.

Feel free to add commits to fix such issues. I will stop rebasing this branch to make it easier for other people to push fixes.

Sage now builds and all doctests pass with this branch. On first sight, the documentation looks good too.

[strogdon]

comment:72

Also, I see two of

! LaTeX Error: Lonely \item--perhaps a missing list environment.

in docpdf.log. After grepping through latex/en/reference it's because of combinat/combinat.tex and graphs/graphs.tex where there is

}, \footnote[2]{\sphinxAtStartFootnote%\begin{enumerate}
\setcounter{enumi}{4}
\item {}
Assmus, J. Key, Designs and their codes, CUP, 1992.

\end{enumerate}

I suspect this should be

}, \footnote[2]{\sphinxAtStartFootnote%
\begin{enumerate}
\setcounter{enumi}{4}
\item {}
Assmus, J. Key, Designs and their codes, CUP, 1992.

\end{enumerate}

which does work. And since this is auto-generated, I suspect a bug?

[jdemeyer]

comment:73

Replying to @strogdon:

And since this is auto-generated, I suspect a bug?

The real problem is that docutils interprets this as a list item (and that's a feature!):

E. Assmus, J. Key, Designs and their codes, CUP, 1992.

This should work:

-.. [2] E. Assmus, J. Key, Designs and their codes, CUP, 1992.
+.. [2] \E. Assmus, J. Key, Designs and their codes, CUP, 1992.

[jdemeyer]

comment:74

See #20496 for this.

[jdemeyer]

Changed dependencies from #20488 to #20496

[jdemeyer]

Changed dependencies from #20496 to #20496, #20498

[sagetrac-git]

Changed commit from f1ae5e6 to e88c802

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

e88c802

Fix malformatted mathematics

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

00722fd	Escape abbreviated first names in references
d92eada	Add documentation
54d3cf8	Clean up multi_modular.pyx
ae6192a	Move multi_modular to arith
bdc3f0b	Merge commit 'd92eadade4c3b6a477a14809adeac3c058458ad3'; commit 'ae6192a2a300f7fb7a1bae3dd8034e4d24767428' into t/18497/ticket/18497

[sagetrac-git]

Changed commit from e88c802 to bdc3f0b

[jhpalmieri]

comment:79

Is this ready for review?

[strogdon]

comment:80

So I've finally been able to build the pdf-docs without error.

[jdemeyer]

comment:81

Let's say it's ready for review then.

[strogdon]

comment:82

This looks good. Everything builds here. I do see

Makefile:27: recipe for target 'all-pdf-ja' failed
make[2]: [all-pdf-ja] Error 255 (ignored)
for f in *.tex; do platex -kanji=utf8  $f; done

when building the Japanese pdfs (the two Japanese pdfs are generated and, as far as I can determine, look OK). Perhaps a consequence of my system Japanese lang installation which took some effort to get it to work.

@jhpalmieri (John) add your name if you concur.

[strogdon]

Reviewer: Steven Trogdon

[jhpalmieri]

Changed reviewer from Steven Trogdon to Steven Trogdon, John Palmieri

[jdemeyer]

comment:84

I never managed to get the Japanese docs working, so I cannot fully test that either.

[vbraun]

Changed branch from u/jdemeyer/ticket/18497 to bdc3f0b