refactor(post/tag): render tag before content

[SukkaW]

What does it do?

Continuation of #3543 (comment) and #4161

Currently post rendering process is something like this: #4161 (comment). But basically, hexo will render markdown first, then the tag (using nunjucks). So hexo has to remove all swig tags before markdown rendering, and restore them after markdown rendering is finished.

In this PR, I make hexo renders tag first, then markdown, so that remove and restore swig tag are not necessary any more, resulted in 6% faster in generation speed. And the post rendering related test cases are passed as well.

I wonder why Hexo has to render tag after markdown at the beginning.

How to test

git clone -b render-tag-before-md https://github.com/sukkaw/hexo.git
cd hexo
npm install
npm test

Screenshots

Pull request tasks

• Add test cases for the changes.
• Passed the CI test.

Issue resolved: #2821

[SukkaW]

I am considering fixing #3346 in this PR as well since that issue is also related with post rendering process.

---

Update:

@stevenjoezhang I have added a test case for #3346. It seems that issue has been fixed.

[stevenjoezhang]

{% pullquote warning %}
Text
{% endpullquote %}
# Title 0
{% pullquote warning %}
Text
{% endpullquote %}

{% pullquote warning %}
Text
{% endpullquote %}

# Title 1

{% pullquote warning %}
Text
{% endpullquote %}

{% pullquote warning %}Text{% endpullquote %}
# Title 2
{% pullquote warning %}Text{% endpullquote %}

{% pullquote warning %}Text{% endpullquote %}

# Title 3

{% pullquote warning %}Text{% endpullquote %}

The bug reported in #4161 was reproduced. I am confirming if this is a bug (or feature) of Nunjucks
https://github.com/mozilla/nunjucks/blob/40dfdf0ea3e6ba43c3a180eca57d935c36d188c2/nunjucks/src/parser.js#L640-L649

[seaoak]

Thank you for this great patch.

I wonder why Hexo has to render tag after markdown at the beginning.

Me, too. 😉

---

However, since #4171 (comment) is still there, we have to find out a way to solve this.

Generally speaking, in Markdown, any block content is separated by empty line(s) each other.

And custom tags in Nunjucks is not intended for Markdown block content only.
So Nunjucks does not generate additional newline characters by default.

That is,

{% pullquote warning %}
Text
{% endpullquote %}
# Title 0
{% pullquote warning %}
Text
{% endpullquote %}

should be written as folows:

{% pullquote warning %}
Text
{% endpullquote %}

# Title 0

{% pullquote warning %}
Text
{% endpullquote %}

If we want to add a newline character automatically for Nunjucks custom tags in Hexo,
the codes in lib/extend/tag.js should be modified as follows:

diff --git a/lib/extend/tag.js b/lib/extend/tag.js
index 31f9f2aa..883fdb5f 100644
--- a/lib/extend/tag.js
+++ b/lib/extend/tag.js
@@ -77,7 +77,7 @@ class NunjucksBlock extends NunjucksTag {
   }

   run(context, args, body) {
-    return this._run(context, args, trimBody(body));
+    return this._run(context, args, trimBody(body)) + '\n';
   }
 }

@@ -111,7 +111,7 @@ class NunjucksAsyncBlock extends NunjucksBlock {
       body = () => result || '';

       this._run(context, args, trimBody(body)).then(result => {
-        callback(err, result);
+        callback(err, result && `${result}\n`);
       });
     });
   }

I'm not sure whether this behavior is desirable in Hexo.
Even so, this modification should be independent of this patch.

[SukkaW]

Even so, this modification should be independent of this patch.

@segayuu

IMHO, it is fine to include this patch inside this PR since it is related with post rendering issue fix.

---

Update:

@stevenjoezhang

I have applied the patch provided by @seaoak and it seems that #4171 (comment) has been fixed.
Related test cases is added as well.

[SukkaW]

Hexo only uses tag.render in post.js, which is after backtick_code_filter is executed.

In backtick_code_filter, { & } are already escape, so it is fine to render with nunjucks directly.

[SukkaW]

This line will be simpler because result must be a string.

@seaoak Updated.

[jiangtj]

Why add \n?

Look at the following test cases, which one is expected?

    const str = [
      'begin',
      '{% test foo bar %}',
      'test content',
      '{% endtest %}',
      'end'
    ].join('\n');
    const result = await tag.render(str);

    // there is only one `\n` before and after
    result.should.eql('begin\nfoo bar test content\nend');
    // Or should there be two `\n` at the end?
    result.should.eql('begin\nfoo bar test content\n\nend');

[seaoak]

This extra \n is to separete a Nunjucks tag from immediate-after content.
So I expect two \n at the end.

Please refer the above comment #4171 (comment)

[stevenjoezhang]

Another issue: #4087
Indented code blocks will be parsed and rendered by Nunjucks.

[SukkaW]

@stevenjoezhang

For indented code blocks there is no better ways. For backtick code blocks we already have a filter.

---

Maybe a indented code filter? Any line started with 4 spaced or \t should be wrapped in <pre><code>?

[seaoak]

@SukkaW @stevenjoezhang

To escape Nunjucks tags in indented code blocks properly,
we might need a new before_post_render filter (same as backtick code blocks)

I try to make a patch of such a filter:
seaoak/hexo:feature/escape_indented_code_block_before_render
https://github.com/seaoak/hexo/tree/feature/escape_indented_code_block_before_render

With merging it, this @SukkaW 's patch resolves the issue #4087.

Do you think about this new filter?
If it is useful, I continue to try to make it. 😄

[SukkaW]

To escape Nunjucks tags in indented code blocks properly,
we might need a new before_post_render filter (same as backtick code blocks)

I'd rather not to do that.

Only escape { and } which is inside code block, and after code block is highlighted.

Also, you can draft a PR (without actually creating it) from your branch.

[SukkaW]

@seaoak

From #4179 we might consider it is necessary for hexo to render markdown first, right?

[seaoak]

@SukkaW Yes. I think so.

[SukkaW]

@seaoak So we should then focus on how to make swig escaping quicker.