Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert directly to docutils without intermediate RST #36

Open
bnaul opened this issue Apr 5, 2016 · 12 comments

Comments

@bnaul
Copy link

@bnaul bnaul commented Apr 5, 2016

Currently there are some markdown formatting difficulties that can arise when converting to HTML. For example, a link like [nbsphinx](https://github.com/spatialaudio/nbsphinx) which should render as nbsphinx (cf. nbsphinx) won't be displayed properly because this text can't easily be represented in the intermediate RST.

Since sphinx can easily produce documents directly from markdown (e.g. using recommonmark), it would be nice if it were possible to skip the RST altogether as this would more reliably reproduce the desired formatting from the notebook. Currently I am using nbconvert to get markdown output and then producing HTML with Sphinx instead of using nbsphinx because of such formatting issues.

cc: @fperez @takluyver

@takluyver

This comment has been minimized.

Copy link
Contributor

@takluyver takluyver commented Apr 6, 2016

Thanks @bnaul . Also pinging @willingc, who works quite a bit on our own docs.

@willingc

This comment has been minimized.

Copy link

@willingc willingc commented Apr 6, 2016

Thanks @takluyver. @bnaul If you have a sample notebook, I would be happy to look at it. In general, there are pros and cons with different conversion workflows. FWIW, we've been using recommonmark and nbsphinx with good success that past few months on Jupyter projects. Also pinging @mgeier for input.

@willingc

This comment has been minimized.

Copy link

@willingc willingc commented Apr 6, 2016

@bnaul Another thing to look at is the exact versions that you are using in Sphinx, nbsphinx, and recommonmark. Certain combinations have more issues than others.

@takluyver

This comment has been minimized.

Copy link
Contributor

@takluyver takluyver commented Apr 6, 2016

To clarify, Fernando and I were just talking to Brett in person, and asked him to open this issue. While there might be workarounds for any specific case, that's a game of whack-a-mole: the non-code parts of notebooks are markdown, so translating to rst before loading notebooks into Sphinx always runs the risk of messing something up.

We suggested that maybe nbsphinx should depend on recommonmark or something similar, and cut the rst stage out of the loading process. I may be able to spare some time to look into this, though I don't know much (anything?) about the Sphinx API.

@willingc

This comment has been minimized.

Copy link

@willingc willingc commented Apr 6, 2016

Ahh...thanks for the context @takluyver. I would be happy to have a direct conversion from markdown in
notebook cells to HTML. The less conversions the better.

@mgeier

This comment has been minimized.

Copy link
Member

@mgeier mgeier commented Apr 6, 2016

Thanks @bnaul for bringing this up and thanks all for the comments!

Quite some time ago, I read that nested inline markup is not possible with docutils. I thought that this is a general limitation of docutils, but as it turns out now, that it is actually only a limitation of the reST parser!

A few weeks ago, @takluyver already mentioned the possibility of using recommonmark: jupyter/nbconvert#222 (comment). At that time I thought this would only be an implementation detail, but as it turns out, using recommonmark may potentially overcome some fundamental problems found in the current implementation.

I started with the intermediate rst stage because it was the only way I could get started at all, but from my standpoint today, I think it would be better to drop the intermediate rst and parse directly into the docutils document structure.

However, experience may show that switching to recommonmark brings more damage than good, but we'll have to try it to find out if that's true.

This will be a lot of work, @takluyver if you are eager to tackle that, feel free to do so.
If not, I can give it a try, but it will take a considerable amount of time ...

@takluyver

This comment has been minimized.

Copy link
Contributor

@takluyver takluyver commented Apr 6, 2016

I wouldn't quite say I'm eager, but I will look into it ;-)

@mgeier

This comment has been minimized.

Copy link
Member

@mgeier mgeier commented Apr 7, 2016

I had another look at the title of this Issue: "Allow conversion via markdown instead of RST" ...

@bnaul If that's literally what you mean, I'm strongly against it!

I'd rather want something like "Convert directly to docutils without intermediate RST".

In this case, nbsphinx would walk the notebook cells, "manually" convert code cells to the docutils representation and use a library like recommonmark to convert Markdown cells.

@takluyver Was that how you understood it?

@bnaul

This comment has been minimized.

Copy link
Author

@bnaul bnaul commented Apr 7, 2016

@mgeier sure, that's a good description; the main issue in my mind is just allowing markdown to be rendered as faithfully as possible.

@takluyver

This comment has been minimized.

Copy link
Contributor

@takluyver takluyver commented Apr 7, 2016

Yep, loading it as directly as possible into the docutils representation would clearly be ideal.

@mgeier mgeier changed the title Allow conversion via markdown instead of RST Convert directly to docutils without intermediate RST Apr 7, 2016
@tbekolay

This comment has been minimized.

Copy link
Contributor

@tbekolay tbekolay commented Apr 12, 2016

I noticed today that including HTML links (<a href="url">txt</a>) in Markdown fails to render the link as Pandoc doesn't preserve HTML links when converting Markdown to RST. This could be considered a bug in Pandoc, but it's also a +1 in favor of this issue :)

@mgeier

This comment has been minimized.

Copy link
Member

@mgeier mgeier commented Apr 27, 2016

Getting tables and math to work might be hard.

There's at least a rudimentary math extension for recommonmark, but I didn't find any documentation about tables.

It's probably less nerve-wrecking to write a new (and extensible) CommonMark parser than trying to fumble all this into recommonmark ...

mgeier added a commit that referenced this issue Aug 6, 2016
Those directives will probably be removed with #36.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.