write markdown amidst nim via a source code filter

[disruptek]

A source code filter literate allows one to write markdown with code blocks and have both the code and the markdown distilled into nim source during compilation.

The patch takes great pains to ensure that the markdown won't be orphaned and can be passed through the compilation in the (several) nim comment formats. Similar care is taken to ensure that line numbers aren't disrupted.

#? literate(style="#[")

# BIG WORDS
## This is some `literate` nim, huh?

    when isMainModule:
      echo "hello world"

Let's **document some more code**:

      proc foo(): string =
        result = "oh how i love string manipulation"

You kinda have to _use your imagination_ if you lack a renderer!

BIG WORDS

This is some literate nim, huh?

when isMainModule:
  echo "hello world"

Let's document some more code:

  proc foo(): string =
    result = "oh how i love string manipulation"

You kinda have to use your imagination if you lack a renderer!

[krux02]

Well, I like it, but I also already have org-mode. Literate programming in org-mode with Nim code snippets is already much more sophisticated than this one. It has nim syntax highlighting for the snippents, and it has many more exporting options than this. The markup language in org-mode also has many more features than markdown.

Don't get me wrong, I think literate programming is cool, but I would rather not have it in the Nim compiler.

If you could introduce a small interface to the compiler, where source code filters like this one could be implemented in a library (compiler plugin/macro), then it would be a different story though.

[disruptek]

I agree with your points; this was the least-impactful patch I could come up with. How would you like source code filters to plug-in?

We should be able to support org-mode easily, and but it isn't relevant to my application -- GitHub doesn't currently allow you to markup your Issues using org-mode. I believe the org-mode support is limited to areas of the site with a heavier editor. For the curious, this is an example of GitHub's org-mode rendering.

[krux02]

I just thought about github issues as well. Would be nice to extract issues from github and verify them automatically. But that means that we need to support much more than just markdown. This would essentially be a markdown flavor of the test spec.

We need to be able to define multiple files with their file name within the markdown document, this is something the test spec if Nim cannot do right now (because existing tests don't have the limitation to be a single file).

[disruptek]

That's what I'm working on building -- you tag an issue and a system turns it into a test and opens, updates, and closes the issue in response to test success and failure.

I'm not certain I follow your comment about defining multiple files, but as this is only a stream processor, we can emit whatever you want, whenever you want. Here's an approach that may give you what you want, though makes the existing pimple of a hack even more unsightly:

We currently have a special case to handle the #? source | code | filter syntax, but we could extend that case to all lines in the input (and not only the first line), which would allow you to "redefine" the filename and comment syntax by specifying a hint to the filter, using a modeline format, like this:

#? literate

    # a filename= argument may be overridden, i suppose?
    discard "more indented source"

#? style="##",filename="anotherfilename.nim"

    # or even just

#? anotherfilename.nim

    discard "more source logically grouped in anotherfilename.nim ???"

This ensures that this filter-specific hint (the #? lines have no indent) is parsed consistently, and it can flow through to the compiler unmodified as well.

[krux02]

I don't think we need to be able to logically group muliple source code blocks into the same file. I know that is what defines literate programming in many situations, but focusing on tests, it is not required. I even think it is better if we don't support it to keep things simple.

[narimiran]

Based on the latest commit (update to devel), I'm guessing you've merged devel into this branch, but this now causes 514 files to be marked as modified and makes code review impossible.

Could you please reset your branch to the previous commit (git reset --hard HEAD^) and then rebase this branch onto the latest devel (git rebase devel)?

[disruptek]

Oops. But the issue isn't code review...