Fix doc syntax highlighting

[jdemeyer]

Currently, the ....: doctest continuation lines are not parsed correctly by Pygments. As a consequence, those lines are not syntax highlighted.

We should also investigate if we can extend Pygments without patching it and whether the highlighting for Sage can be pushed upstream.

Depends on #19968

CC: @slel

Component: packages: standard

Issue created by migration from https://trac.sagemath.org/ticket/19972

[jhpalmieri]

comment:1

I think this has been solved. src/sage/docs/conf.py has the lines

# Default lexer to use when highlighting code blocks, using the IPython
# console lexers. 'ipycon' is the IPython console, which is what we want
# for most code blocks: anything with "sage:" prompts. For other IPython,
# like blocks which might appear in a notebook cell, use 'ipython'.
highlighting.lexers['ipycon'] = IPythonConsoleLexer(in1_regex=r'sage: ', in2_regex=r'[.][.][.][.]: ')
highlighting.lexers['ipython'] = IPyLexer()
highlight_language = 'ipycon'

and in particular, in2_regex defines ....: as syntax for an input prompt. The documentation looks okay to me: here for example.

[jhpalmieri]

comment:2

I propose closing as a duplicate of #27528.

[fchapoton]

comment:3

how can I check that this works ?

[jhpalmieri]

comment:4

I posted a link to part of Sage's documentation in comment:1, in which the docstrings starting with "....:" are highlighted correctly (as far as I can tell). You can also search for "....:" in Sage sources, and then look in your local copy of the reference manual to see if those lines are highlighted. You could also make the change

diff --git a/src/sage/docs/conf.py b/src/sage/docs/conf.py
index bf2024ce5b..502289b0dd 100644
--- a/src/sage/docs/conf.py
+++ b/src/sage/docs/conf.py
@@ -156,7 +156,7 @@ pygments_style = 'sphinx'
 # console lexers. 'ipycon' is the IPython console, which is what we want
 # for most code blocks: anything with "sage:" prompts. For other IPython,
 # like blocks which might appear in a notebook cell, use 'ipython'.
-highlighting.lexers['ipycon'] = IPythonConsoleLexer(in1_regex=r'sage: ', in2_regex=r'[.][.][.][.]: ')
+highlighting.lexers['ipycon'] = IPythonConsoleLexer()
 highlighting.lexers['ipython'] = IPyLexer()
 highlight_language = 'ipycon'
 

and rebuild Sage and its reference manual (make doc-clean then make) to see what effect that line has. I did this, and the highlighting on lines starting either "sage:" or "....:" disappears.

[jhpalmieri]

comment:5

More evidence: if you look at the html source, with the highlighting code in place, "sage:" and "....:" are labeled as, for example:

<span class="gp">....: </span>

and looking at the file basic.css, "gp" stands for "generic prompt". Without the highlighting code (that is, if you make the change in the previous comment), the whole line instead is marked class="go", and "go" stands for "generic output".

I suppose we could write a doctest for this, but I don't have the energy right now.

[fchapoton]

comment:6

ok, thanks. Look good to me.