SG incorrectly grabbing the description when a label is defined in docstring

[choldgraf]

I noticed that SG is incorrectly grabbing the description when you define a label in the docstring before the title. E.g.:

"""
.. my_label:

=======
Title
=======

etc etc.
"""

SG will correctly find title, but when it tries to pull the description it shows up as =====Title===== because it's splitting on the wrong line.

For example see my matplotlib (new) tutorials build here: http://predictablynoisy.com/matplotlib/tutorials/index.html

[Titan-C]

I'm surprised Sphinx-Gallery builds with docstrings which start with anything else that the title. Our parser is not as strict as I thought. Sphinx-Gallery is grabbing the label as title and the title as the first paragraph when generating the gallery thumbnails. We don't believe in putting anything before the title of the python example in the docstring, it does not make much sense for an independent file. You don't need to put your own labels for each example, Sphinx-Gallery does it for you. Our documentation is certainly not so evident on this but is described in this section: Know your gallery files. https://sphinx-gallery.readthedocs.io/en/latest/getting_started.html#know-your-gallery-files where we reference the introduction example. You can also see it in use from this example: https://sphinx-gallery.readthedocs.io/en/latest/auto_examples/sin_func/plot_sin.html#sphx-glr-auto-examples-sin-func-plot-sin-py In summary to reference any example you use :ref:`sphx_glr_path_to_example_with_every_path_separation_as_underscore.py` If still in doubt just check your generated .rst files of the gallery to find the generated label each one has.

[choldgraf]

yeah - we had a debate about this with the MNE team a little while back, but there was pushback from people who didn't want to have such long labels. I don't think that we realized that this wasn't supported.

Kind of a bummer because that means we need to go back through many many matplotlib examples that already have a label in them and switch it to the long sphinx-gallery version :-/ that said, I can understand why you don't want to support this. In that case, it'd be helpful if there was an error any time the first line of the docstring wasn't part of a title, instead of it working but silently pulling the wrong text for description.

[choldgraf]

One final thought on this: one reason we've had disagreements about using the sphinx gallery links etc is that they can be really, really long. If you're using something like a PEP checker that's forcing all of your lines to be <= 79 characters, this means that one sphinx gallery link can take up like 80% of the length of a line.

That said, I got all of the links etc working for matplotlib's docs, so it's not a dealbreaker. I just think it's worth considering. I'll close the issue tho if it's not something worth implementing! (I promise I'm not trying to be confrontational, I just don't like dangling issues that don't have anything actionable in them :-) )

[Titan-C]

It's true nobody likes to write long reference names. And maybe for some examples that are often called it might be good to have your personal reference, for all the others I do believe the automatically generated is fine. This issue nevertheless made me think about how robust our parser is or how strict we want it to be. Maybe I went the path of over complicating the solution to this issue(A simple work around is to ignore anything in the docstring starting with ".." because that is RST comment, and it will work). I used docutils to have a proper parser for the RST in the syntax and use that to extract the first paragraph for the tooltip. I works well in isolation but it completely crashes when executed inside a Sphinx build, because Sphinx has a nasty way of overloading docutils. Thus I did not open a PR but I put my attempt in https://github.com/Titan-C/sphinx-gallery/commits/docstring. For reference. I'm for the moment unsure if it is worth digging further on the RST parser for the docstrings and extract our first paragraph for the tooltip there is no easy good solution. I would nevertheless say that this component of Sphinx-Gallery is particularly brittle. We don't deem it as high priority because there is an easy work around from the users side. But issues are maybe accumulating: #136, #222. Also going deeper into an RST parse might also help with issue #219, since from PR #220 going through pandoc to do the parsing is not enough. @lesteve, @GaelVaroquaux any comments?

[choldgraf]

I should say that right now I am coming from the perspective of a person trying to convert all of the matplotlib docs over to sphinx-gallery...so I am probably not representative of the doc situation in most packages. Just trying to share my experience thus far :D

At the very least, I think it's worth adding some kind of warning or callout in the docs for SG that says "note: you should not put anything before the title in the docstring" so that users don't think this is a bug like I did :)

[lesteve]

Sorry I am just looking at this now ...

Just trying to share my experience thus far :D

This is super useful, thanks a lot for this, keep the comments coming our way !

In this particular case, it does feel like we way we find the title and description for the tooltip is too fragile.

Since you seem to say that the title is found correctly maybe we can just remove the title from the docstring and find the paragraph once the title has been removed. This would make the whole thing a little less brittle.

Side-note: I agree that parsing the rst would be a better way to do it, I don't know too much about sphinx and sphinx monkey-patching docutils certainly does not encourage me to look further into it ...

[choldgraf]

Since you seem to say that the title is found correctly maybe we can just remove the title from the docstring and find the paragraph once the title has been removed. This would make the whole thing a little less brittle.

I'm a +1 for this, though also understand if the preferred behavior is to just raise an error if it finds anything in the docstring above the title. The main thing to me is that it shouldn't silently do the wrong thing :-)

[larsoner]

@choldgraf I think this has been fixed in some PR (one of mine I think?), too

[choldgraf]

yep! thanks for squashing these issues