Support fenced code block within lists and blockquotes

[waylan]

As per this discussion on the markdown list, we should support fenced code blocks inside lists and blockquotes. Currently, they only work at the document root.

While we're at it, we might add support for github's syntax as an alternative??

[waylan]

Just a reminder to myself: Lots of test cases in these bug reports.

[rafalp]

Anything new on this issue? I'm using this lib in my project and allowing users to quote messages with fenced code blocks is pretty important for me. Maybe directions how can this be handled by custom extension?

[cgabard]

Hello,

I have the same issue in a personal project and solved it with a dirty hook : change the BlockParser.parseChunk() method with an extension. This hook allow some preprocessor extensions, in this case only fenced_code, to process text blocks before there are split arount blank line.

A cleaner way will be to have a new kind of extension, or add an configuration option to preprocessor extension, which allow their to be processed on each texts block.

I can submit you a PR if I know how you want I implemented this.

[waylan]

@cgabard thanks for the input. Although I have a plan. Some time ago (around the time I filed this issue) I made a small change to the BlockParser (b3e0359) which will allow fenced code blocks to be implemented as a BlockProcessor.

While BlockProcessors generally only work on one block at a time, they have access to all remaining (unprocessed) blocks in the document. Given the above mentioned change, a BlockProcessor's 'run' method could step through all remaining blocks until the end of the fenced code block is found. If there is no end to the fenced code block, the run method would return False and the BlockParser would move on just as is the test method had returned False.

The tricky part is when the fenced code is nested in a blockquote, list, etc. If we reach the end of the nested level, but haven't found the end of the fenced code block, then no fenced code block. That means keeping better track of nesting (how many levels deep, etc) which means the existing BlockProcessors probably need some refactoring to. And while I'm at it, those BlockProcessors might as well be refactored to use the new API mentioned above (it would greatly simplify a few of them). Its that last part that has been holding this up. Refactoring the BlockProcessors is a major task which I just don't have a big enough block of spare time to tackle.

Of course, if someone else wanted to give it a go, I'd be willing to mentor them through it.

[cgabard]

thanks for the reply.

It is a major piece of work but I agree it will be a better solution to solve this issue. Unfortunately, I don't have time for now to help you on this big task. But, meanwhile, I can do some smaller task for help you.

[memeplex]

+1 for this:

• github markdown supports it.
• the codehilite extension also supports it. Nowadays I can 8-space indent a codeblock and annotate it with #!python or :::python but not with ~~~~{.python}.

This is somewhat inconsistent inside python markdown itself and, moreover, it's not possible to preview some markdown before pushing it to github.

[aziz]

Just a reminder that the fenced code blocks are not working inside admonition blocks.

!!! note "This is the note title"
    Here is the sample code

    ``` js
    for (var i = 0; i < 10; i++) {
        console.log(i);
    }
    ```

[dhalperi]

+1 here as well.

@cgabard @waylan if you have some pointers about your plan and/or example similar code, I may be able to take a stab at the new implementation. I have a long plane flight coming up... :)

[dhalperi]

(My particular issue is in the attached gist: https://gist.github.com/dhalperi/5d0f7c8fc780a14495a4).

Specifically, that line 27 does not render the same as line 12, which works fine. In the attached screenshot, line 27 is the first code block under List item 2. Line 12 is the second code block not in the list.

[waylan]

Every single block processor needs to be refactored first (as per my previous comment). Only then can we even start to implement fenced code blocks. Each blockparser's test method should return True as a deprecation path (when they are all refactored we can deprecate the test method). Then, each run method needs to take the given block, either parse it (and return True) or leave the block intact and return False. Some of the blockparser's will have multiple return False statements nested inside if statements throughout, while others will be more simple. If you want to work on this, start with a few simpler ones and do a pull request. I'll be happy to offer a review.

[facelessuser]

I have a workaround that works pretty well for this. This is not a proposed official solution, but a pretty good workaround. I attempted to re-factor fenced blocks as a block processor, and though I kind of got it working, there were a number of problems in lists etc. As waylan mentioned earlier, there is a ton of work that would need to be done to get this working for the official solution as waylan has planned. I frankly, don't have the motivation for that kind of effort right now, but I did want something reliable working for nested fences now.

So I re-factored fenced blocks to work more like a procedural parser that can find fenced blocks anywhere (even in blockquotes). It should handle generally well in block quotes with the lazy way it requires > characters (but I haven't stressed it to any great extent yet):

    A blockquote
    > A quote
    >
    > - list
    >     some words
    >     some more words
    >
    >     - sub list
    >
    >         ```python
    >         a = re.compile(r"""\d +  # the integral part
    >            \.    # the decimal point
    >            \d *  # some fractional digits""", re.X)
    >         b = re.compile(r"\d+\.\d*")
    >         ```

    >
    >  > Sub blockquote with spaced out text
    >  > ```python
    >    def displaymatch(match):
    >
    >  >     if match is None:

    >

    >  >         return None
    >  > return '<Match: %r, groups=%r>' % (match.group(), match.groups())
    >  > ```
    >

In some cases, code blocks try and eat the html paceholders for the nested fenced blocks since fenced blocks are still done as a preprocessor and code blocks are blockprocessors. For example, in the event that someone has formatted their lists improper, and the list item falls back to text, the code block processor can try to gobble up fenced block's html placeholders. For that reason, I had to also modify the code blockprocessor to be aware of them and not eat them. It seems to work pretty good.

Anyways, final results (that is fenced blocks nested under lists which are nested under an admonition blocks 😄):

The entire thing is confined to a single extension that extends fenced blocks to being nested (for anyone who cares to to try it): https://github.com/facelessuser/PyMdown/blob/master/pymdown/nestedfences.py

I imagine in time I will get around to stressing it better, but in general it seems to work pretty good for me. It seems so far to work pretty well as a stopgap solution. Anyways take it for what it's worth, just thought I'd share it.

EDIT: Python Markdown 2.5.X required.

EDIT: Readjusted algorithm to restore fences if they were too greedy so you can included fenced syntax in an indented block properly again. Can now also turn off indented blocks if desired. Also fixed some internal indentation issues with fenced blocks.

[cgabard]

For those interested, I have an other ugly workaround which work well. It is an extension allowing to use any preprocessors like a blockprocessor. It modifie parseChunk parser method with a new one which run specified pre-processors before others block-process. As parseChunk is called by others block processors to process inner content, it work well for fenced code in others blocks like quote, liste or whatever block.

The source is here : https://gist.github.com/cgabard/c0df670a816c80ac04ea

It is a very ugly hack buck it work well and can work with other pre-processors like footnote or reference which then can be defined inside blockquote.

I use it like this preprocess_ext = PreprocessBlockExtension({"preprocess" : ("fenced_code_block", "footnote", "reference","abbr", )})

[facelessuser]

@cgabard, out of curiosity, I tired yours (had to reactor for Python Markdown 2.5.X), but it did not work. Maybe it only works on Python Markdown < 2.5.X?

Made be realize I should say that mine will only work for >= 2.5.X. Probably should have said that up front. It could be refactored for lower versions, but as I am on 2.5.X, it made sense to code it for that.

Anyways, hopefully between the couple of workarounds posted, people can get their nested fenced blocks working until a more polished, official approach is available.

[cgabard]

Maybe, I use it with 2.4
Le 2 oct. 2014 17:29, "Isaac Muse" notifications@github.com a écrit :

@cgabard https://github.com/cgabard, out of curiosity, I tired yours
(had to reactor for Python Markdown 2.5.X), but it did not work. Maybe it
only works on Python Markdown < 2.5.X?

Made be realize I should say that mine will only work for >= 2.5.X.
Probably should have said that up front. It could be refactored for lower
versions, but as I am on 2.5.X, it made sense to code it for that.

Anyways, hopefully between the couple of workarounds posted, people can
get their nested fenced blocks working until a more polished, official
approach is available.

—
Reply to this email directly or view it on GitHub
#53 (comment)
.

[dregad]

I'd love to see this fixed.

[Argoday]

+1

[waylan]

I've locked this issue as +1's add no value to the conversation. We already know what needs to be done to fix this and it is on the top of my list. This issue will be resolved in version 3.0. See the roadmap for status.