Intersphinx for cvxopt, cvxpy, cypari2, cysignals, flint, fpylll, gmpy2, ipywidgets, matplotlib, mpmath, networkx, numpy, rpy2, scipy, sympy #37575

mkoeppe · 2024-03-09T22:54:04Z

Intersphinx allows us to refer to functions and classes in other projects using the standard Sphinx roles :func:, :class:, ...

Examples in the documentation preview:

Based on a rebased version of #29231 by @mwageringel. In addition to the scipy intersphinx done in #29231, here we add a number of relevant Python libraries, as well as flint, whose docs are built with Sphinx as well.

To address concerns in the discussion there about the docbuild contacting remote servers for the inventory files, here we instead vendor the inventory files:

$ ls -l src/doc/common/_vendor
-rw-r--r--  1 mkoeppe  staff    1942 Mar  9 21:20 cvxopt.inv
-rw-r--r--  1 mkoeppe  staff   13251 Mar  9 21:20 cvxpy.inv
-rw-r--r--  1 mkoeppe  staff   10728 Mar  9 21:20 cypari2.inv
-rw-r--r--  1 mkoeppe  staff     775 Mar  9 21:20 cysignals.inv
-rw-r--r--  1 mkoeppe  staff  246266 Mar  9 21:20 flint.inv
-rw-r--r--  1 mkoeppe  staff    1603 Mar  9 21:20 fpylll.inv
-rw-r--r--  1 mkoeppe  staff    2891 Mar  9 21:20 gmpy2.inv
-rw-r--r--  1 mkoeppe  staff   10934 Mar  9 21:20 ipywidgets.inv
-rw-r--r--  1 mkoeppe  staff  105887 Mar  9 21:20 matplotlib.inv
-rw-r--r--  1 mkoeppe  staff    3115 Mar  9 21:20 mpmath.inv
-rw-r--r--  1 mkoeppe  staff   51830 Mar  9 21:20 networkx.inv
-rw-r--r--  1 mkoeppe  staff   78006 Mar  9 21:20 numpy.inv
-rw-r--r--  1 mkoeppe  staff    1449 Mar  9 21:20 pplpy.inv
-rw-r--r--  1 mkoeppe  staff  136166 Mar  9 21:20 python.inv
-rw-r--r--  1 mkoeppe  staff    3289 Mar  9 21:20 rpy2.inv
-rw-r--r--  1 mkoeppe  staff  112691 Mar  9 21:20 scipy.inv
-rw-r--r--  1 mkoeppe  staff   55596 Mar  9 21:20 sympy.inv

This extends our existing practice with the Python inventory (moved here from its previous location src/doc/common/python3.inv). The new command sage -python -m sage_docbuild.vendor retrieves the latest versions of these files. (Distribution notes: These files are not installed, as they are only needed at the build time of the documentation. After #36730, they are part of the sdist of sagemath-doc-html, but not part of the wheel.)

By setting SAGE_DOC_REMOTE_INVENTORIES=yes, this can be overridden, as previously suggested in #29231 (comment). In this case it is first tried to contact the remote servers.

Fixes #29231 (stalled since 2020).
Fixes #27164 (stalled since 2019).

Outside the scope of this PR:

adding such Intersphinx links everywhere (that would be a long-term writing project - https://groups.google.com/g/sage-devel/c/PfYpuOWt8xQ/m/Emn7pZd5AQAJ)
linking to documentation of non-Sphinx projects (see Sphinx extlinks for ecl, gap, giac, maxima, meson, ppl, qepcad, scip, singular, soplex #37598, Package breathe - Sphinx plugin for project documentation using Doxygen #37577)
any considerations how these .inv files could/should/would be taken from our installations of these packages, or from system packages that ship the documentation.

📝 Checklist

The title is concise and informative.
The description explains in detail what this PR is about.
I have linked a relevant issue or discussion.
I have created tests covering the changes.
I have updated the documentation accordingly.

⌛ Dependencies

… not contact remotes; use vendored inventories for scipy, flint

…T function

…ventories.sh

…x references

dcoudert · 2024-03-13T17:48:54Z

src/sage/calculus/desolvers.py

@@ -27,8 +27,8 @@
  order equations, return list of points.

 - :func:`desolve_odeint` - Solve numerically a system of first-order ordinary
-  differential equations using ``odeint`` from `scipy.integrate module.
-  <https://docs.scipy.org/doc/scipy/reference/integrate.html#module-scipy.integrate>`_
+  differential equations using :func:`~scipy:scipy.integrate.odeint` from


I'm not sure to understand the meaning of the ~. Here we have :func:`~scipy:scipy.integrate.odeint`, but line 1502 you use :func:`scipy:scipy.integrate.odeint`. Why and why is it needed ?

The ~ gives short links, such as "odeint()" instead of "scipy.integrate.odeint()"

Here I use it because the text already says from what module it comes.

dcoudert · 2024-03-13T17:50:58Z

src/sage/matrix/matrix_double_dense.pyx

@@ -594,8 +594,8 @@ cdef class Matrix_double_dense(Matrix_numpy_dense):

        ALGORITHM:

-        Computation is performed by the ``norm()`` function of
-        the SciPy/NumPy library.
+        Computation is performed by the :func:`~scipy:scipy.linalg.norm`


again, here we have a ~ but not line 742.

dcoudert · 2024-03-14T17:55:19Z

src/sage/graphs/generic_graph.py

+              ``'graphml'``           | :func:`networkx.readwrite.graphml.write_graphml`
+              ``'multiline_adjlist'`` | :func:`networkx.readwrite.multiline_adjlist.write_multiline_adjlist`
+              ``'pajek'``             | :func:`networkx.readwrite.pajek.write_pajek`
+              ``'yaml'``              | :func:`networkx.readwrite.nx_yaml.write_yaml`


When I check on the deployed documentation, this link is not working
https://deploy-preview-37575--sagemath.netlify.app/html/en/reference/graphs/sage/graphs/generic_graph#sage.graphs.generic_graph.GenericGraph.export_to_file

I think this function simply does not exist any more in current versions of networkx. Is it tested anywhere?

It is no longer in networkx (https://networkx.org/documentation/stable/reference/readwrite/index.html).

It's not tested in method export_to_file and according to the code of the method, it is no longer in the list of considered formats. I think it has been removed sometime ago and we have forgotten to remove it from the documentation.

Should I remove it here?

Done in a55ee28

src/sage/graphs/generators/families.py

…'yaml'

github-actions · 2024-03-15T22:36:32Z

Documentation preview for this PR (built with commit a55ee28; changes) is ready! 🎉

dcoudert

I have checked several links in the generated documentation and it looks good.

However, I have not checked what may happen without Internet connection for both the construction of the documentation and its consultation on a local installation. Should someone check the before accepting this PR ?

mkoeppe · 2024-03-16T16:32:58Z

Thanks!

I'll try right now, so let me disconnNO CARRIER

mkoeppe · 2024-03-16T17:21:31Z

Tested: It builds correctly without access to the Internet, the HTML pages render correctly without access, obviously clicking the links to the external documentation give an error page, but after enabling access to the Internet, the links work.

dcoudert

Thanks for testing. I think it's an interesting feature.

LGTM.

mkoeppe · 2024-03-16T18:34:56Z

Thanks for the review!

…ppl, qepcad, scip, singular, soplex       We create a standard way to refer to the online manuals of these packages, which are generated by tools other than Sphinx and therefore cannot be linked to using Intersphinx. This extends what has been already done with Pari. Examples: - [Examples in the Developer Guide](https://deploy-preview-37598-- sagemath.netlify.app/html/en/developer/sage_manuals#hyperlinks) Fixes sagemath#27495. Outside of the scope of this PR: - adding such extlinks everywhere - working with upstream projects to upgrade to better documentation tools - projects that can be linked to using Intersphinx; that's sagemath#37575 - any consideration how to link instead to local copies of those manuals, whether installed from SPKG or system packages. ### 📝 Checklist  - [x] The title is concise and informative. - [x] The description explains in detail what this PR is about. - [x] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation accordingly. ### ⌛ Dependencies    URL: sagemath#37598 Reported by: Matthias Köppe Reviewer(s): Kwankyu Lee, Matthias Köppe, Max Horn

mkoeppe added c: documentation c: build labels Mar 9, 2024

mkoeppe changed the title ~~Intersphinx for scipy~~ Intersphinx for scipy and flint Mar 10, 2024

mkoeppe changed the title ~~Intersphinx for scipy and flint~~ Intersphinx for cvxopt, cvxpy, cypari2, cysignals, flint, fpylll, gmpy2, ipywidgets, matplotlib, mpmath, networkx, numpy, rpy2, scipy, sympy Mar 10, 2024

mkoeppe self-assigned this Mar 10, 2024

mkoeppe force-pushed the intersphinx_scipy branch from fd24f9f to b20e5b4 Compare March 10, 2024 06:31

mkoeppe added the s: needs review label Mar 10, 2024

mkoeppe requested review from dcoudert and mezzarobba March 10, 2024 07:22

mkoeppe added 17 commits March 10, 2024 16:34

29231: add intersphinx mapping for scipy

00e8b6f

29231: fallback for scipy inventory file when not connected to internet

cf0d818

29231: use online pplpy docs for intersphinx with local fallback

48f95b9

src/doc/common/python3.inv: Update

4ff50cd

src/doc/common/update-inventories.sh: Rename from update-python-inv.sh

1c021dd

src/sage_docbuild/conf.py: Update pplpy doc url

645377b

src/doc/common/_vendor: Move python.inv here

bf2879d

src/sage_docbuild/conf.py: Unless SAGE_DOC_REMOTE_INVENTORIES=yes, do…

8de1104

… not contact remotes; use vendored inventories for scipy, flint

src/sage/stats/basic_stats.py: Update intersphinx reference to scipy

3156b95

src/sage_docbuild/conf.py: Actually set the intersphinx_mapping

fb7fe0d

src/doc/common/update-vendored-inventories.sh: Add fpylll.inv

b3b47e2

src/sage_docbuild/conf.py: Fix, refactor

1de4c58

src/sage/libs/flint/arith_sage.pyx: Add intersphinx reference to FLIN…

1abd2f5

…T function

sage_docbuild.vendor: New, replaces src/doc/common/update-vendored-in…

1bee07c

…ventories.sh

src/sage_docbuild/conf.py: Add more intersphinx items

700e40f

src/sage_docbuild/conf.py: Add some more intersphinx items

b880a48

src/sage/libs/flint/arith_sage.pyx: Fix markup

4c0c20c

mkoeppe force-pushed the intersphinx_scipy branch from b20e5b4 to 4c0c20c Compare March 10, 2024 23:34

mkoeppe added 3 commits March 10, 2024 17:01

src/sage/graphs: Replace broken networkx.lanl.gov links by intersphin…

b4fe463

…x references

src/doc/en/developer/sage_manuals.rst: Document intersphinx

363797b

src/doc/en/developer/sage_manuals.rst: Expand intersphinx examples

99f4d0a

mkoeppe mentioned this pull request Mar 12, 2024

Sphinx extlinks for ecl, gap, giac, maxima, meson, ppl, qepcad, scip, singular, soplex #37598

Merged

5 tasks

dcoudert reviewed Mar 13, 2024

View reviewed changes

dcoudert suggested changes Mar 14, 2024

View reviewed changes

mkoeppe added 2 commits March 14, 2024 11:58

src/sage/graphs/generators/families.py: Fix networkx references

93b9987

GenericGraph.export_to_file: Update documentation for removed format=…

a55ee28

…'yaml'

dcoudert reviewed Mar 16, 2024

View reviewed changes

dcoudert approved these changes Mar 16, 2024

View reviewed changes

dcoudert added s: positive review and removed s: needs review labels Mar 16, 2024

vbraun merged commit 6406386 into sagemath:develop Mar 31, 2024
18 checks passed

mkoeppe added this to the sage-10.4 milestone Mar 31, 2024

github-actions bot removed the s: positive review label Apr 1, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Intersphinx for cvxopt, cvxpy, cypari2, cysignals, flint, fpylll, gmpy2, ipywidgets, matplotlib, mpmath, networkx, numpy, rpy2, scipy, sympy #37575

Intersphinx for cvxopt, cvxpy, cypari2, cysignals, flint, fpylll, gmpy2, ipywidgets, matplotlib, mpmath, networkx, numpy, rpy2, scipy, sympy #37575

mkoeppe commented Mar 9, 2024 •

edited

dcoudert Mar 13, 2024

mkoeppe Mar 13, 2024 •

edited

mkoeppe Mar 13, 2024

dcoudert Mar 13, 2024

dcoudert Mar 14, 2024

mkoeppe Mar 14, 2024

dcoudert Mar 15, 2024

mkoeppe Mar 15, 2024

dcoudert Mar 15, 2024

mkoeppe Mar 15, 2024

github-actions bot commented Mar 15, 2024

dcoudert left a comment

mkoeppe commented Mar 16, 2024

mkoeppe commented Mar 16, 2024

dcoudert left a comment

mkoeppe commented Mar 16, 2024

Intersphinx for cvxopt, cvxpy, cypari2, cysignals, flint, fpylll, gmpy2, ipywidgets, matplotlib, mpmath, networkx, numpy, rpy2, scipy, sympy #37575

Intersphinx for cvxopt, cvxpy, cypari2, cysignals, flint, fpylll, gmpy2, ipywidgets, matplotlib, mpmath, networkx, numpy, rpy2, scipy, sympy #37575

Conversation

mkoeppe commented Mar 9, 2024 • edited

📝 Checklist

⌛ Dependencies

Choose a reason for hiding this comment

mkoeppe Mar 13, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Mar 15, 2024

dcoudert left a comment

Choose a reason for hiding this comment

mkoeppe commented Mar 16, 2024

mkoeppe commented Mar 16, 2024

dcoudert left a comment

Choose a reason for hiding this comment

mkoeppe commented Mar 16, 2024

mkoeppe commented Mar 9, 2024 •

edited

mkoeppe Mar 13, 2024 •

edited