The rst contents directive does not work with sagenb (wrong anchors)

[sagetrac-tmonteil]

The following RST directive does not work in the notebook:

.. contents:: Table of contents

The table of contents are created, but the anchors do not correspond, titles are replaced bu IDs.

This bug seems not new but i do not find it on the trac.

CC: @nthiery @seblabbe @sagetrac-tmonteil @dimpase

Component: notebook

Issue created by migration from https://trac.sagemath.org/ticket/17371

[kcrisman]

comment:1

Can you be more specific about what you mean by "does not work"? Are you using rst2sws or just uploading an rst file, or ... ?

(I ask, but of course I don't know much about ReST directives, so I'm not sure I can help you immediately.)

Then I can report this "upstream" as well.

[sagetrac-tmonteil]

comment:2

The problem appears when i upload the rst file directly from the notebook.

For the table of contents directive, you can have a look at http://docutils.sourceforge.net/docs/ref/rst/directives.html#table-of-contents

Now i tried to translate it into a sws file first, and i get the following error:

$ sage -rst2sws file.rst file.sws

Traceback (most recent call last):
  File "/opt/sagemath/sage-6.3/src/bin/sage-rst2sws", line 136, in <module>
    W = nb.create_new_worksheet(title, 'admin')
  File "/opt/sagemath/sage-6.3/local/lib/python2.7/site-packages/sagenb-0.11.1-py2.7.egg/sagenb/notebook/notebook.py", line 459, in create_new_worksheet
    if username!='pub' and self.user_manager().user_is_guest(username):
  File "/opt/sagemath/sage-6.3/local/lib/python2.7/site-packages/sagenb-0.11.1-py2.7.egg/sagenb/notebook/user_manager.py", line 184, in user_is_guest
    return self.user(username).is_guest()
  File "/opt/sagemath/sage-6.3/local/lib/python2.7/site-packages/sagenb-0.11.1-py2.7.egg/sagenb/notebook/user_manager.py", line 112, in user
    return self._user(username)
  File "/opt/sagemath/sage-6.3/local/lib/python2.7/site-packages/sagenb-0.11.1-py2.7.egg/sagenb/notebook/user_manager.py", line 544, in _user
    raise LookupError("no user '{}'".format(username))
LookupError: no user 'admin'

This problem seems unrelated to the contents directive. Not sure whether i have to open another ticket for that.

[kcrisman]

comment:3

My guess on the original problem is that https://github.com/sagemath/sagenb/blob/master/sagenb/notebook/docHTMLProcessor.py isn't handling this properly. I wonder if that might also happen with an html document with intra-document links? If you could run docutils on your rst file and then upload that html file to the notebook and see what happens, that could also be useful. Or post a minimal rst file that doesn't work.

Sebastien, looks like you wrote the rst converter - any ideas?

---

I get the same error for the sage -rst2sws - apparently after processing most of it, because I get error messages for stuff when I put one that has roles it doesn't recognize. Perhaps no one has used this recently and there was some change in the notebook that caused problems.

Once again, Sebastien, I think you wrote this - any ideas? Sorry for the double question, just trying to figure this out and I haven't looked closely at either of these things before.

from sage.misc.misc import tmp_dir
nb = Notebook(tmp_dir()+'.sagenb')

# create a worksheet
W = nb.create_new_worksheet(title, 'admin')

Maybe new notebooks no longer have a default admin user. I can confirm that this did work in Sage 5.2, but already was broken in Sage 5.13... That only narrows it down between sagenb-0.9.1 and 0.10.7.2, unfortunately.

[kcrisman]

comment:4

Okay, I figured out the (very) short answer to the second question. At some point new users were no longer created for a notebook, apparently. It's no problem to fix, though.

diff --git a/src/bin/sage-rst2sws b/src/bin/sage-rst2sws
index 0a7b92b..33e7721 100755
--- a/src/bin/sage-rst2sws
+++ b/src/bin/sage-rst2sws
@@ -131,6 +131,7 @@ worksheet_txt = translator.process_doc_html(html)
 from sagenb.notebook.notebook import Notebook
 from sage.misc.misc import tmp_dir
 nb = Notebook(tmp_dir()+'.sagenb')
+nb.user_manager().create_default_users('password')
 
 # create a worksheet
 W = nb.create_new_worksheet(title, 'admin')

---

But I won't make a branch for that quite yet. Looking at it a bit more, the original problem is really with the refs. (I discovered this from the errors I got in a successful run of rst2sws and then what happened when I uploaded it.) Namely, ref and toctree are not docutils ReST. They are Sphinx. One possible workaround is given here but probably one will have to use the `label`_ markup to get this to work right. Possibly this is a wontfix? :-(

[seblabbe]

comment:5

Bonjour Karl, Thierry,

• At Fix broken rst2sws and add doctests for the scripts rst2txt and rst2sws #13297, I had the 2 years old project of adding tests for cmd line rst2txt and rst2sws. If I had done this, it would have prevented the second issue raised in this ticket. Thanks for the report, you motivated me to add those doctests (I also fixed the script as you suggested). It now needs review.

• I was able to reproduce the table of content problem in three ways from the Upload page using three different files : file.rst, file.txt and file.sws. The file.rst is below. The file.sws was obtained from sage -rst2sws. The file.txt was obtained from sage -rst2txt.

• The file.rst I used for my test is:

****
Tile
****

.. contents:: Table of contents

First Section
=============

:: 

    sage: 3+4
    7

Second Section
==============

:: 

    sage: 8
    8

• The output of sage -rst2txt file.rst file.txt (with Fix broken rst2sws and add doctests for the scripts rst2txt and rst2sws #13297) contains the problem. But the problem seems to really come from docutils. Compare href="#first-section" with href="#id1":

sage: rst = open('file.rst','r').read()
sage: from docutils.core import publish_string
sage: html = publish_string(rst, writer_name='html')
sage: print html
...
<h1 class="title">Tile</h1>

<div class="contents topic" id="table-of-contents">
<p class="topic-title first">Table of contents</p>
<ul class="simple">
<li><a class="reference internal" href="#first-section" id="id1">First Section</a></li>
<li><a class="reference internal" href="#second-section" id="id2">Second Section</a></li>
</ul>
</div>
<div class="section" id="first-section">
<h1><a class="toc-backref" href="#id1">First Section</a></h1>
<pre class="literal-block">
sage: 3+4
7
</pre>
</div>
<div class="section" id="second-section">
<h1><a class="toc-backref" href="#id2">Second Section</a></h1>
<pre class="literal-block">
...

• This is using sage-6.4 and docutils version 0.7 which is quite old. The ticket docutils-0.12 #16733 was merged in sage-6.5.beta0 which updates docutils to version 0.12. We have to check if that table of content issue is fixed in 0.12.

sage: import docutils
sage: docutils.__version__
'0.7'

[kcrisman]

comment:6

Indeed, in 6.5.beta0

sage: docutils.__version__
'0.12'

but I can confirm that there is still the same thing. But I don't think this is actually problematic. It even says back-references. Right?

• The first table entry has id id1, so that the first section heading links back to it.
• The first table entry anchor links to first-section, which is the id of the div around that section.
• The second ones behave similarly.
  So I would argue this file is behaving correctly! I tried it with a longer file (attached) and it works correctly.

Unless I am missing something. However, when I do something with :ref: directives (the Sphinx, not docutils, one), I get errors like

$ ./sage -rst2txt Programming.rst Programming.txt
<string>:18: (ERROR/3) Unknown interpreted text role "ref".
<string>:20: (ERROR/3) Unknown interpreted text role "ref".
<string>:22: (ERROR/3) Unknown interpreted text role "ref".
<string>:24: (ERROR/3) Unknown interpreted text role "ref".
<string>:26: (ERROR/3) Unknown interpreted text role "ref".
* Unknown command \ZZ

[kcrisman]

Attachment: file.rst.gz

[kcrisman]

comment:7

And yes it is true that docutils is just putting these things in order with id numbers. So in that sense intra-doc links will not work properly, if they pre-existed - I guess one has to use Sphinx, I don't know? But that might be a bit much to combine with sagenb.

[mkoeppe]

comment:8

outdated, should be closed