Files not included in TOC tree with .rst files are used as index / header

[lhoupert]

Describe the bug
Hi all,
I am building a jupyter-book but some of my files are not included in the toc tree.

To Reproduce
Steps to reproduce the behavior:

1. Go to 'https://github.com/lhoupert/from-python-to-numpy/tree/jb-version', clone the directory and cd jb-version
2. Build the jupyter-book jupyter-book build .

Expected behavior

My _toc.yml file is:

- file: 00-book

- header: Get started
- file: 01-preface
- file: 02-introduction
- file: 03-anatomy

- header: Vectorization
- file: 04-code-vectorization
- file: 05-problem-vectorization
- file: 06-custom-vectorization
- file: 07-beyond-numpy

- header: Conclusion
- file: 08-conclusion

- header: References
- file: 09-quick-reference
- file: 10-bibliography

but when the html is built a few sections are missing:

You can see the whole website here

This is consistent with the warnings I got when building the website:

/Users/locupe/Dropbox/Work/Python/Repos_useful/from-python-to-numpy/jb-version/00-book.rst:81: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/Users/locupe/Dropbox/Work/Python/Repos_useful/from-python-to-numpy/jb-version/00-book.rst:91: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/Users/locupe/Dropbox/Work/Python/Repos_useful/from-python-to-numpy/jb-version/00-book.rst:110: WARNING: Explicit markup ends without a blank line; unexpected unindent.

checking consistency... /Users/locupe/Dropbox/Work/Python/Repos_useful/from-python-to-numpy/jb-version/02-introduction.rst: WARNING: document isn't included in any toctree
/Users/locupe/Dropbox/Work/Python/Repos_useful/from-python-to-numpy/jb-version/03-anatomy.rst: WARNING: document isn't included in any toctree
/Users/locupe/Dropbox/Work/Python/Repos_useful/from-python-to-numpy/jb-version/05-problem-vectorization.rst: WARNING: document isn't included in any toctree
/Users/locupe/Dropbox/Work/Python/Repos_useful/from-python-to-numpy/jb-version/06-custom-vectorization.rst: WARNING: document isn't included in any toctree
/Users/locupe/Dropbox/Work/Python/Repos_useful/from-python-to-numpy/jb-version/07-beyond-numpy.rst: WARNING: document isn't included in any toctree
/Users/locupe/Dropbox/Work/Python/Repos_useful/from-python-to-numpy/jb-version/10-bibliography.rst: WARNING: document isn't included in any toctree

Interestingly, when I didnt put any header entries in the _toc.yml file, the html rendering only display the first two from my TOC.

You will also notice something strange happening on the first page of the website, the name of the files which were not included in the toc are listed:

Another problem with the warning errors below. They don't make sense as my file 00-book.rst has only 74 lines...

from-python-to-numpy/jb-version/00-book.rst:81: WARNING: Explicit markup ends without a blank line; unexpected unindent.
from-python-to-numpy/jb-version/00-book.rst:91: WARNING: Explicit markup ends without a blank line; unexpected unindent.
from-python-to-numpy/jb-version/00-book.rst:110: WARNING: Explicit markup ends without a blank line; unexpected unindent.

Thank you for the help!
Environment (please complete the following information):

• Python Version [Python 3.7.4]
• Output of Jupyter Book: 0.7.1

Additional context
Add any other context about the problem here.

[choldgraf]

Hmmm - this feels like a bug with how the toctree objects are generated when the book is built with rST pages. I wonder if we need to add an extra line at the end of this rST toctree template here: https://github.com/executablebooks/jupyter-book/blob/master/jupyter_book/toc.py#L84

Any chance you wanna try giving that a shot to see if it fixes your problem? Happy to give pointers...

[lhoupert]

Thank you @choldgraf for the quick reply! I am away for the new 3 weeks but I can work on this when I come back!

[choldgraf]

I think this may have been fixed by #768 - can you try again from master and let me know if it works? thanks!

[lhoupert]

Hi @choldgraf I installed the master branch of version of jupyter-book using

pip install https://github.com/executablebooks/jupyter-book/archive/master.zip  

So now my version is

Jupyter Book: 0.7.2dev0

But it didn't fix the problem. Once I have a little bit of time next month, I will play with the toc.py script you mentioned. Any advices on where to start?

Thank you!

[phaustin]

@lhoupert -- it would helpful to capture this as an MRE in https://github.com/executablebooks/jupyter-book/tree/master/tests/books/toc using the same content layout as that folder (or augmenting it if necessary to reproduce the bug)

@choldgraf -- along those lines -- is there a worked-example of a numbered toc with the new master? I've been sprinkling numbered: true in various places in https://github.com/executablebooks/jupyter-book/blob/master/tests/books/toc/_toc_withheaders.yml but it's not behaving as I expected (i.e. I can't seem to number all the sections)

[choldgraf]

Mmm - it should just be a matter of adding numbered: true to a top-level section in _toc.yml. Can you make sure to also upgrade to the latest sphinx-book-theme?

[lhoupert]

@lhoupert -- it would helpful to capture this as an MRE in https://github.com/executablebooks/jupyter-book/tree/master/tests/books/toc using the same content layout as that folder (or augmenting it if necessary to reproduce the bug)

@choldgraf -- along those lines -- is there a worked-example of a numbered toc with the new master? I've been sprinkling numbered: true in various places in https://github.com/executablebooks/jupyter-book/blob/master/tests/books/toc/_toc_withheaders.yml but it's not behaving as I expected (i.e. I can't seem to number all the sections)

@phaustin Thank you for your advices. I am quite new in all of this. Can you explain me what is an MRE?

I just tested with the last version of jupyter-book and the problem is still here.

jupyter-book --version                                                                                                        ✭ 15:12:25
Jupyter Book: 0.7.4
MyST-NB: 0.8.4
Sphinx Book Theme: 0.0.35
MyST-Parser: 0.9.0
Jupyter-Cache: 0.2.2

[phaustin]

Sorry -- MRE means "minimal reproducible example" -- i.e. the simplest table of contents that doesn't work. We haven't merged the fix for this yet so you'll need to update your jupyter-book install like this:

pip install git+https://github.com/phaustin/jupyter-book@posix_path --upgrade --no-deps

[lhoupert]

@phaustin Thank you for the help. I updated my install following your recommendation but the problem is still here.

I managed to reproduce the bug using the content layout from the folder you indicated.

To reproduce the bug, I add to create an index file at the .rst format. The content of the index_rst.rst file can be seen below:

Main Index
==========

.. toctree::
 :maxdepth: 2

Then, I was able to reproduce the bug with this _toc.yml file:

file: index_rst
title: Toc
sections:
- file: content3
- file: content3
- file: content3

[lhoupert]

Hi @phaustin and @choldgraf, do you have suggestions on how to move things forward?

[lhoupert]

Hi guys,

I tried with the last version of jupyter-book but still the same problem persists. Additional information, I realised this is working if the first file (index) is a mardown file and not an .rst. However if I am using an .rst file as the first file of a section or part, it doesnt work. Here is another example of _toc.yml (that doesn't work) using the files from this repository: https://github.com/executablebooks/jupyter-book/tree/master/tests/books/toc

- file: content3

- part: Test1
  chapters:
  - file: content3
  - file: content3
  - file: content3

- part: Test2
  chapters:
  - file: content3
  - file: content3
  - file: content3

[AakashGfude]

Thanks, @lhoupert for the additional information. Quite strange, as it is happening only if the index file is an rst. Will do some more tests today.

[AakashGfude]

Another problem with the warning errors below. They don't make sense as my file 00-book.rst has only 74 lines...

from-python-to-numpy/jb-version/00-book.rst:81: WARNING: Explicit markup ends without a blank line; unexpected unindent.
from-python-to-numpy/jb-version/00-book.rst:91: WARNING: Explicit markup ends without a blank line; unexpected unindent.
from-python-to-numpy/jb-version/00-book.rst:110: WARNING: Explicit markup ends without a blank line; unexpected unindent.

I suspect these warnings are indicating the extra lines (name of the files not listed in toc) as you showed in this picture :

[lhoupert]

Thank you @AakashGfude for having a look. I updated a minimal example here:
https://github.com/lhoupert/jupyter-book/tree/test-rst-toc/tests/books/toc/test_rst

The _toc.yml file is:

- file: content0

- part: Part1
  chapters:
  - file: content3
  - file: content3a

- part: Part2
  chapters:
  - file: content3b
  - file: content3c
  - file: content3d

When running jupyter-build in this directory. I don't have the warning WARNING: Explicit markup ends without a blank line; unexpected unindent but I got these error messages:

checking consistency... /Users/locupe/Dropbox/Work/Python/Repos_fork/jupyter-book/tests/books/toc/test_rst/content3a.rst: WARNING: document isn't included in any toctree
/Users/locupe/Dropbox/Work/Python/Repos_fork/jupyter-book/tests/books/toc/test_rst/content3c.rst: WARNING: document isn't included in any toctree
/Users/locupe/Dropbox/Work/Python/Repos_fork/jupyter-book/tests/books/toc/test_rst/content3d.rst: WARNING: document isn't included in any toctree

[lhoupert]

It seems that it is fixed now! thanks to #1293 .

The only thing I add to do was to migrate my pre-existing toc to the new format (I just had to follow the instructions in my terminal after trying to build with the last version of master).