format docstrings

[bramtayl]

Ok, this should be mostly working now. Style and state should propagate to embedded code chunks. The only remaining issue is a problem with \\ (see the one new test broken) that I was hoping someone else would understand.

[bramtayl]

Supercedes #225

[domluna]

So the failed case has to do with string escaping and unescaping, which is one of the reasons the string is read directly from the file and not via .val field of the CSTParser node. I think the markdown parser is doing something similar.

It would be best to check with some other example docstrings where escaping/unescaping would alter the output.

[bramtayl]

I wonder if this is a bug in Markdown.plain? To be honest once it gets to 4 backslashes I get kinda lost

"""
`\\` \\\\
"""
test(x) = x
Markdown.plain(@doc test) # doesn't match text above

[bramtayl]

Also, I'm very confused about the format check failure...not sure how to fix the formatting when I'm changing the formatting simultaneously...

[domluna]

You have to run format() on the JuliaFormatter.jl directory.

It's mostly there as a sanity check reminder to make sure the change hasn't caused unwanted changes in the files. Sort of like an integration test.

[bramtayl]

Ok got it. Well that seems to error, which is strange because the tests pass...I'll do some digging

[bramtayl]

Bad news: I definitely introduced a bug that wasn't caught by the tests. Good news: I found an extra test that reveals the bug.

[bramtayl]

@domluna I can't figure out what's causing the bug, do you have any ideas?

[domluna]

I'll take a look tomorrow. Probably related to docstrings being eaten somehow though

[bramtayl]

@domluna sorry to bother you again. Just looking forward to very pretty docstrings. Did you get a chance to look at this?

[domluna]

feel free to bug me as much you like 🙂 . Could you rebase off master and push again?

Btw, regarding the rendering engine thing https://github.com/MichaelHatherly/CommonMark.jl looks promising but I haven't done a deep dive with it yet.

[mortenpi]

Just a drive-by comment, which may or may not be useful: In Documenter we need to do something similar when we want to update the doctest outputs automatically. The approach we have taken is to just regex for the code blocks and then replace the content between the backticks. So we don't ever parse the Markdown, getting around the issue of the re-created Markdown not being identical to the original. And it seems to work quite reliably.

[bramtayl]

I think there's pluses and minuses of leaving the markdown part alone. On one hand, it guarantees round-trip-ability and is a lot simpler; on the other hand, there is potential to make markdown prettier too. Happy with either

[domluna]

example of hooking into CommonMark to format julia code blocks MichaelHatherly/CommonMark.jl#4 (comment)

[bramtayl]

Ooh that's nice!

[bramtayl]

Rebased as #247