File content is listed without liquid processing, if its name goes after the listing file

[thybzi]

• I believe this to be a bug, not a question about using Jekyll.
• I am on (or have tested on) Windows 10+
• It was another bug.

My Reproduction Steps

1. Clear jekyll, latest version (3.6.0 for now), without any plugins
2. Create an 'listing' page (e.g. foo.html) containing a loop for site.pages and outputting their {{ page.content }}
3. Create any other page with filename which is alphabetically "later" than that index one (e.g. foo0.html, fop.html, google.html or jazz.md etc.).
4. Also create some page "prior" filename (e.g. bar.html, baz.md or fon.html etc.)
5. Both 'later' and 'prior' should have any liquid constuct inside content ({% include %}, {% assign %} etc.)
6. Open 'listing' page in browser

The Output I Wanted

When opening 'index' page in browser, I expect to see both 'prior' and 'later' pages content with liquid constructs processed.
Actually, liquid contructs are only processed in 'prior pages content', and left 'as is' (e,g, unprocessed) in 'later' pages.

Details:

1. Reproduces in a similar way on local Windows 10 and GitHub Pages production (see "4." below for the only difference I've found)
2. Doesn't depend on page url (permalink), or directory nesting. Depends on filename only
3. If there are multiple 'listing' pages in project, the filename of the first of them (alphabetically) becomes the 'breakpoint' (e.g. if you have 'foo.html' and 'fred.html' with similar site.pages outputting their page.content, fop.html will have its content output unprocessed in both pages)
4. If there is a file with the same name as 'listing' page, but resides in another directory, its content is output unprocessed on github pages production, though remains UNprocessed on local Windows 10

[thybzi]

My bug description seems to be a little bit complicated, so see the code and the demo:
http://github.com/thybzi/jekyll-issue-6465
http://thybzi.github.io/jekyll-issue-6465/foo.html

[thybzi]

Seems to be some kind of output-loop prevention.
Though made in a strange way.

[Crunch09]

This occurs because the Renderer hasn't parsed the liquid in those files yet.

My proposal to fix this would be to store the information on the Page if its content has been parsed yet and if not parse it on demand if its content is requested. What do you think @jekyll/build ? cc: @ashmaroli @pathawks

[ashmaroli]

My proposal to fix this would be to store the information on the Page if its content has been parsed yet and if not parse it on demand if its content is requested.

That seems like a complicated flow to me.. if there are multiple listing pages (edge-case) then there might be instances of multiple parse-on-demand calls.. leading to future addition of caching of previously-parsed-content and checks....

IMO, a "listing" page ought to be specially designated and as such generated towards the end of the build process.. similar to how jekyll-feed generates a feed.xml using a low-priority Generator.

[parkr]

I think we need to take the approach of on-demand generation. When a Liquid drop requests a Page/Document’s content, it should receive the compiled output. We have a dependency chain mapped a little in our regenerator, but it isn’t fully fleshed out yet. I was talking with @mneumegen about making regeneration on-demand, and this seemed like a good approach to us.

/cc @jekyll/performance

[Crunch09]

@parkr Just so i understand do you mean if i use site.pages in my page all the pages in my site should be added as a dependency to the current page? (like it is used right now for the include tag and layouts)

[parkr]

@Crunch09 Yes!

[thybzi]

Not stale, do you need anything from me to fix/reproduce this?

[DirtyF]

Reproduced on latest Jekyll, any help welcome to make sure all files are processed before outputing the index file.

[rvansa]

I am having a similar problem though in my case the pages are not processed in a strictly alphabetic order. Also it's not only about rendering liquid - markdown is not rendered into HTML either.
It seems that adding a front matter (can be empty) fixes the ordering, though I am a bit reluctant to modifying all the pages (and generators for reference).
Similar issue here: https://talk.jekyllrb.com/t/accessing-page-content-that-contains-liquid-statements/3506

[agowa]

I've a similar problem. But for me it's within _includes. I'm using lunr.

This is the code I use within _includes\search-lunr.html which is included within the _layouts\default.html.

Therefore I think the part with dependencies @parkr mentioned isn't working (anymore) either.

{% assign counter = 0 %}
var documents = [{% for page in site.pages %}{% if page.url contains '.xml' or page.url contains 'assets' %}{% else %}{
    "id": {{ counter }},
    "url": "{{ page.url | absolute_url }}",
    "title": "{{ page.title }}",
    "body": "{{ page.content | markdownify | replace: '\','\\\\' | replace: '.', '. ' | replace: '</h2>', ': ' | replace: '</h3>', ': ' | replace: '</h4>', ': ' | replace: '</p>', ' ' | strip_html | strip_newlines | replace: '  ', ' ' | replace: '"', ' ' }}"{% assign counter = counter | plus: 1 %}
    }, {% endif %}{% endfor %}{% for page in site.without-plugin %}{
    "id": {{ counter }},
    "url": "{{ page.url | absolute_url }}",
    "title": "{{ page.title }}",
    "body": "{{ page.content | markdownify | replace: '\','\\\\' | replace: '.', '. ' | replace: '</h2>', ': ' | replace: '</h3>', ': ' | replace: '</h4>', ': ' | replace: '</p>', ' ' | strip_html | strip_newlines | replace: '  ', ' ' | replace: '"', ' ' }}"{% assign counter = counter | plus: 1 %}
    }, {% endfor %}{% for page in site.posts %}{
    "id": {{ counter }},
    "url": "{{ page.url | absolute_url }}",
    "title": "{{ page.title }}",
    "body": "{{ page.date | date: "%Y/%m/%d" }} - {{ page.content | markdownify | replace: '\','\\\\' | replace: '.', '. ' | replace: '</h2>', ': ' | replace: '</h3>', ': ' | replace: '</h4>', ': ' | replace: '</p>', ' ' | strip_html | strip_newlines | replace: '  ', ' ' | replace: '"', ' ' }}"{% assign counter = counter | plus: 1 %}
    }{% if forloop.last %}{% else %}, {% endif %}{% endfor %}];

[ashmaroli]

@agowa338 Are you using Jekyll 4.2.0?
If yes, I suggest trying again using a Gemfile to point to the master branch of this repository:

gem 'jekyll', github: 'jekyll/jekyll'

If that's not possible, try downgrading to Jekyll 4.1.1

[agowa]

Yes, I was using Jekyll 4.2.0. I now also tried master as you suggested and it works with that one.