New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] module/package level pragma `{.doctype: markdown|rst.}` to help migrate docs from RST to Markdown #9291

Open
timotheecour opened this Issue Oct 10, 2018 · 18 comments

Comments

Projects
None yet
10 participants
@timotheecour
Collaborator

timotheecour commented Oct 10, 2018

There seems to be a consensus (or near-consensus) that markdown is preferred over RST for documentation, see https://forum.nim-lang.org/t/4280 (Drop RST and join the Markdown train?) and that the only valid argument for keeping RST is historical reason.

A number of people have also noted that supporting a hybrid will just cause more issues, so let's explore practical ways to gradually migrate docs to markdown.
To ease the migration I propose the following:

introduce a module/package-level annotation {.doctype: markdown|rst.} which states how to interpret documentation comments (as either markdown or RST depending on doctype).

usage

foo/
  nim.cfg # contains --doctype:markdown : applies to all modules under foo/
  foo_bar1.nim
  foo_bar2.nim # can override nim.cfg with `{.doctype: markdown|rst.}`
bar3.nim
bar4.nim # contains `{.doctype: markdown.}`

so doctype would only have to be specified at a few places (eg just 1, in nimble/nim.cfg/config.nims) until this becomes the new default.

automating documentation source code migration from rst to markdown

  • tooling should be able to handle the 99% case fairly easily, the rest could be manual fixups; tooling should report when it can't transform a doc comment, and when the resulting markdown is invalid; this could be part of nim doc, nimpretty, or a separate tool.

@timotheecour timotheecour changed the title from RFC module/package level pragma `{.doctype: markdown|rst.}` to help migrate docs from RST to Markdown to [RFC] module/package level pragma `{.doctype: markdown|rst.}` to help migrate docs from RST to Markdown Oct 10, 2018

@Araq

This comment has been minimized.

Member

Araq commented Oct 11, 2018

How do I write

`term`:idx:

in markdown?

@alaviss

This comment has been minimized.

Contributor

alaviss commented Oct 11, 2018

You'd need pandoc-flavored markdown for it: https://pandoc.org/try/?text=%60term%60%3Aidx%3A&from=rst&to=markdown

@dom96

This comment has been minimized.

Member

dom96 commented Oct 11, 2018

If we're going to move to a new format then I vote for asciidoc ;)

@Araq

This comment has been minimized.

Member

Araq commented Oct 11, 2018

Ok, so it's just some poorly thought-out proposal causing lots of friction, yay.

@andreaferretti andreaferretti added the RFC label Oct 12, 2018

@arnetheduck

This comment has been minimized.

Contributor

arnetheduck commented Oct 15, 2018

How do I write

you don't. this, along with the phone book, has been deprecated in the modern world in favor of search.

@Araq

This comment has been minimized.

Member

Araq commented Oct 15, 2018

Oh really, but Nim's search uses :idx:.

@arnetheduck

This comment has been minimized.

Contributor

arnetheduck commented Oct 17, 2018

Oh really, but Nim's search uses :idx:.

how.. quaint ;)

@timotheecour

This comment has been minimized.

Collaborator

timotheecour commented Oct 21, 2018

can we start with gradual migration of rst files to markdown?

Here's a typical example where rst falls short:

https://raw.githubusercontent.com/nim-lang/Nim/devel/doc/codeowners.rst renders as https://github.com/nim-lang/Nim/blob/devel/doc/codeowners.rst

see also rendering:
image

an entire table is missing (Compiler table), probably due to some obscure easy to miss syntax error noone noticed.

This happened to me with rst files in other context, where code snippets were missing.

@GULPF

This comment has been minimized.

Member

GULPF commented Oct 21, 2018

@timotheecour To be fair, that's only due to differences between githubs and Nims RST parsers. It renders fine on the website. That problem wouldn't go away by replacing RST - markdown is notoriously ill-specified, especially when it comes to non-standard features like tables.

IMO if Nim is going to switch to markdown, there first must be a pure Nim implementation. Preferable one that implements a well specified form of markdown like CommonMark (note however that commonmark does not support anything fancy like tables).

@timotheecour

This comment has been minimized.

Collaborator

timotheecour commented Oct 21, 2018

I think it's fair to assume a lot of people will browse through sources via github ( eg https://github.com/nim-lang/Nim/blob/devel/doc/codeowners.rst) so silently skipping entire sections (tables here, code blocks in other cases I ran into) is a bit of an issue...

note however that commonmark does not support anything fancy like tables

tables need to be supported...

That problem wouldn't go away by replacing RST - markdown is notoriously ill-specified

IIRC markdown will at least always render something (even if badly), but it won't skip anything, as is the case with RST as domonstrated above.

@citycide

This comment has been minimized.

Contributor

citycide commented Oct 21, 2018

@GULPF

IMO if Nim is going to switch to markdown, there first must be a pure Nim implementation

Luckily, @soasme is working on one 😄 https://github.com/soasme/nim-markdown

@soasme

This comment has been minimized.

soasme commented Oct 22, 2018

Thanks for the citing, @citycide. And Thanks again for @dom96 merging the package publishing PR the other days.

In the latest version bumping (v0.3.0), I've added the table support. In the next few days, I'll add more test cases to make it more stable. It could be awesome if nim-markdown can be used.

@Araq

This comment has been minimized.

Member

Araq commented Oct 22, 2018

an entire table is missing (Compiler table), probably due to some obscure easy to miss syntax error noone noticed.

Yeah, RST sucks which is why Nim's RST parser deviates from the standard... And yeah, markdown has the same problems.

@dom96

This comment has been minimized.

Member

dom96 commented Oct 22, 2018

@soasme I'll happily use it on the Nim forum. :)

@timotheecour

This comment has been minimized.

Collaborator

timotheecour commented Oct 22, 2018

/cc @Araq

And yeah, markdown has the same problems

  • well tables are supported by both github's markdown as well as https://github.com/soasme/nim-markdown/

  • furthermore, markdown won't silently skip something it can't understand and will isntead render it the best it can (and then when readers see the bad rendering, it's easy to spot there's a mistake) ; whereas rst will just skip over and it can be hard to spot there's a mistake in the 1st place, such as the example I gave with a missing table)

@Araq

This comment has been minimized.

Member

Araq commented Oct 23, 2018

So what, markdown has significant trailing whitespace. I mean seriously, it's all pretty bad.

@timotheecour

This comment has been minimized.

Collaborator

timotheecour commented Oct 23, 2018

has significant trailing whitespace

we don't have to use features we don't like and we can easily write a linter to reject that (eg in a github hook / in CI).

instead of using trailing whitespace for newline, we can just use the popular extension escaped_line_breaks.
github supports it and probably (EDIT: now also) nim-markdown, so we can write instead:

this is line1\
this is line2
@krux02

This comment has been minimized.

Contributor

krux02 commented Oct 28, 2018

Well I would like to add my two cents. Documentation in Nim is written in neither markdown nor RST. It is written in whatever the documentation parser understands. And the documentation parser understands an RST/Markdown hybrid. So people care write their comments in whatever they want to write it in.

Enforcing a specific style of doc comments is not something for a pragma, it is the job of a linter. So my vote for the pragma is a no.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment