New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Req: "Order of interpretation" for http://jekyllrb.com/docs #5808

Closed
swizca opened this Issue Jan 22, 2017 · 12 comments

Comments

Projects
None yet
7 participants
@swizca

swizca commented Jan 22, 2017

  • This is a feature request.
  • I am on (or have tested on) Debian/Ubuntu GNU/Linux

Could an article on order of interpretation be added, please?

It is confusing keeping what's doing what, particularly when running into problems.

e.g. pseudo

process_html(
  process_javascript(
    process_markdown(
      process_liquid(
        process_yaml(
          <file>
)))))

?

  • it doesn't help that serve/build shows liquid errors with line numbers based on stripped out yaml / Front Matter. (i.e. hard to know which line number the error message really means.)

Further, could this article outline how to see the output of a page / each individual step?

e.g. It seems that kramdown comes along for the (jekyll) installation ride, which is cool, one can kramdown file | less if one wanted to see how one garbled that aspect, however it is not apparent how one captures what one is providing to kramdown when one builds any particular page.

(Thus the ability to build only one page, to individually see how one has mucked up something they're actively working on, would be very useful.)

I find I run into this need mostly when I have botched some liquid. I can examine the result in the _site/.html page, but I cannot review, say, the intermediary .md file. gigo applies, of course. (But without the in between bits one's gi becomes go^x by the time the .html results, if it does at all.)

Thanks for listening.

@DirtyF

This comment has been minimized.

Show comment
Hide comment
@DirtyF

DirtyF Jan 22, 2017

Member

Hi,

👍 for more documentation on how to debug layouts

According to renderer.rb the order is:

  1. Liquid
  2. Kramdown
  3. Layout

You might want to read How Jekyll works that also deals with debugging with pry.

@jekyll/build should be able to answer more precisely on this particular matter.

Member

DirtyF commented Jan 22, 2017

Hi,

👍 for more documentation on how to debug layouts

According to renderer.rb the order is:

  1. Liquid
  2. Kramdown
  3. Layout

You might want to read How Jekyll works that also deals with debugging with pry.

@jekyll/build should be able to answer more precisely on this particular matter.

@swizca

This comment has been minimized.

Show comment
Hide comment
@swizca

swizca Jan 22, 2017

Thank you very much for that link.

It seems to confirm that I do indeed understand this jekyll eco-system, reducing that nagging itch that I'm missing something. Thank you for that.

'course it doesn't take into account nested includes and layouts, but one gets the idea via the article.

I believe I wrote the OP when deeply frustrated with trying to figure out which is causing what. IIRC, something like one thing causing excess <p>{stuff}</p> or <code></p>\n<p>```</p><p><code>{more stuff}</p>\</code> .... and on and on. Ultimately solved, that moment, by putting a space between ``` and {.

Being able to process just one page, or have intermediate files saved to /tmp/_site or something would help know who to blame. It can be hard to tell which beastie is twitching at that particular moment - be it liquid or kramdown or YAML. The subtle differences between (or lack of) [invisible] white space can be quite exasperating. (Never mind tabs vs spaces, and one's usual indentation style for maintainer clarity.)

Terrifically useful at the link is seeing the intermediary step between .md and .html (the latter nicely visible under _site when debugging). I didn't know liquid was injecting raw html - good to know. (The {% highlight %} example provided a D'OH! moment. As in ... of course it must be injecting raw html, and therefore kramdown must be processing it ok. 'Huh.', silently, to myself.)

swizca commented Jan 22, 2017

Thank you very much for that link.

It seems to confirm that I do indeed understand this jekyll eco-system, reducing that nagging itch that I'm missing something. Thank you for that.

'course it doesn't take into account nested includes and layouts, but one gets the idea via the article.

I believe I wrote the OP when deeply frustrated with trying to figure out which is causing what. IIRC, something like one thing causing excess <p>{stuff}</p> or <code></p>\n<p>```</p><p><code>{more stuff}</p>\</code> .... and on and on. Ultimately solved, that moment, by putting a space between ``` and {.

Being able to process just one page, or have intermediate files saved to /tmp/_site or something would help know who to blame. It can be hard to tell which beastie is twitching at that particular moment - be it liquid or kramdown or YAML. The subtle differences between (or lack of) [invisible] white space can be quite exasperating. (Never mind tabs vs spaces, and one's usual indentation style for maintainer clarity.)

Terrifically useful at the link is seeing the intermediary step between .md and .html (the latter nicely visible under _site when debugging). I didn't know liquid was injecting raw html - good to know. (The {% highlight %} example provided a D'OH! moment. As in ... of course it must be injecting raw html, and therefore kramdown must be processing it ok. 'Huh.', silently, to myself.)

@tomjoht

This comment has been minimized.

Show comment
Hide comment
@tomjoht

tomjoht Jan 26, 2017

Contributor

@swizca I'll write up a doc page for this. can you provide some scenarios where it's important to know the order of interpretation?

for example, if you have a variable in your layout file, your page can't read that variable because your page renders first before it gets stuffed into the layout.

another example - in your include, suppose you have a variable. you insert that include into a page. if the variable is defined in the page's frontmatter, does that variable contained in the include get rendered?

or conversely, suppose you have a variable in your page that is referenced by the include. does the include render the page's variable?

what kind of scenarios become clear when you understand the order of interpretations? I don't have a clear sense of what you were trying to debug.

also, does the basic rendering process of (1) liquid, (2) markdown, (3) layout occur with each page and each include individually, or does it happen sitewide across all pages and includes? for example, process all liquid in every file first, then process all markdown in every file.

It might be good to point out that some files don't follow this rendering process at all:

  • yaml files in the data folder won't render markdown or liquid
  • yaml frontmatter won't render liquid or markdown

Finally, would it be helpful to include the tabbed display of the layout transformation that this author does in the post referenced earlier in this thread?

Contributor

tomjoht commented Jan 26, 2017

@swizca I'll write up a doc page for this. can you provide some scenarios where it's important to know the order of interpretation?

for example, if you have a variable in your layout file, your page can't read that variable because your page renders first before it gets stuffed into the layout.

another example - in your include, suppose you have a variable. you insert that include into a page. if the variable is defined in the page's frontmatter, does that variable contained in the include get rendered?

or conversely, suppose you have a variable in your page that is referenced by the include. does the include render the page's variable?

what kind of scenarios become clear when you understand the order of interpretations? I don't have a clear sense of what you were trying to debug.

also, does the basic rendering process of (1) liquid, (2) markdown, (3) layout occur with each page and each include individually, or does it happen sitewide across all pages and includes? for example, process all liquid in every file first, then process all markdown in every file.

It might be good to point out that some files don't follow this rendering process at all:

  • yaml files in the data folder won't render markdown or liquid
  • yaml frontmatter won't render liquid or markdown

Finally, would it be helpful to include the tabbed display of the layout transformation that this author does in the post referenced earlier in this thread?

@swizca

This comment has been minimized.

Show comment
Hide comment
@swizca

swizca Jan 26, 2017

Those all sound good. 😁

I found the tabbed display very helpful.

I no longer remember the specifics that made me send the OP, but I do remember becoming very, very, frustrated with trying to figure out why I was seeing what I was, having put in A but getting D, knowing there was at least B and C in between ... or could it possibly be C then B.

Some things you could add to the list:

  • when interpreters are called. e.g. I couldn't figure out why (it turns out) markdown in my .liquid file wasn't being processed. (D'uh!) [Rename to .md and all references ...]
  • why yes, html can go in markdown!
  • but no, <p>{%assign var=val%}</p> won't do what you expect, but <a>{%assign var=val%}</a> will. (Native block vs inline html tags.)
  • Front Matter can be counter-productive, e.g. {%include%}, .css, .md, sccsified or markdownified, even.
  • <!-- {%comment%}{::comment} sometimes invisible{:/comment}, but not {::comment/}, sometimes not {%endcomment%} will drive you nuts keeping straight as to result -->
  • HTML still matters, css will drive you nuts
  • add # of lines of front matter and the liquid error line # may make sense.
  • browser inspectors are your friend. So are IDE's (file 'group' editors)
  • javascript and JSON are easier than they look - one doesn't have to do a deep dive to get the bit of usefulness needed.
  • it's easy to lose sight that, in the end, all one is providing is some includes to a browser, be it the HTML page, css, or scripts. The browser will then chew on them further to display what you see.
  • Under the hood there is still DOM ...

One comes to Jekyll with the impression ... write some markdown, magic happens. And it does.

Inevitably, one will think ... Well what if I ... made that red or bigger or this or that ...
(And you'll make copies of files for backup just in case, and try this, and that, and that didn't work so what about ... feels like files are like rabbits.)

Pointers to best practices for _config.yaml would be good. Having played around and experimented with it all (Jekyll) for a while, I found my _config.yaml incoherent and inconsistent.

- part of that was coming to this with no knowledge of Ruby or Gemfiles or 'bundle'.

swizca commented Jan 26, 2017

Those all sound good. 😁

I found the tabbed display very helpful.

I no longer remember the specifics that made me send the OP, but I do remember becoming very, very, frustrated with trying to figure out why I was seeing what I was, having put in A but getting D, knowing there was at least B and C in between ... or could it possibly be C then B.

Some things you could add to the list:

  • when interpreters are called. e.g. I couldn't figure out why (it turns out) markdown in my .liquid file wasn't being processed. (D'uh!) [Rename to .md and all references ...]
  • why yes, html can go in markdown!
  • but no, <p>{%assign var=val%}</p> won't do what you expect, but <a>{%assign var=val%}</a> will. (Native block vs inline html tags.)
  • Front Matter can be counter-productive, e.g. {%include%}, .css, .md, sccsified or markdownified, even.
  • <!-- {%comment%}{::comment} sometimes invisible{:/comment}, but not {::comment/}, sometimes not {%endcomment%} will drive you nuts keeping straight as to result -->
  • HTML still matters, css will drive you nuts
  • add # of lines of front matter and the liquid error line # may make sense.
  • browser inspectors are your friend. So are IDE's (file 'group' editors)
  • javascript and JSON are easier than they look - one doesn't have to do a deep dive to get the bit of usefulness needed.
  • it's easy to lose sight that, in the end, all one is providing is some includes to a browser, be it the HTML page, css, or scripts. The browser will then chew on them further to display what you see.
  • Under the hood there is still DOM ...

One comes to Jekyll with the impression ... write some markdown, magic happens. And it does.

Inevitably, one will think ... Well what if I ... made that red or bigger or this or that ...
(And you'll make copies of files for backup just in case, and try this, and that, and that didn't work so what about ... feels like files are like rabbits.)

Pointers to best practices for _config.yaml would be good. Having played around and experimented with it all (Jekyll) for a while, I found my _config.yaml incoherent and inconsistent.

- part of that was coming to this with no knowledge of Ruby or Gemfiles or 'bundle'.

@pathawks

This comment has been minimized.

Show comment
Hide comment
@pathawks

pathawks Jan 26, 2017

Member

It seems like this issue is moving toward nore general frustration than an actionable bug report.

I can empathize with being new to Jekyll and not understanding all of the components, but lets try to keep the scope of this issue narrow enough that there is a clear action that needs to be taken. Otherwise, there is always https://talk.jekyllrb.com

Member

pathawks commented Jan 26, 2017

It seems like this issue is moving toward nore general frustration than an actionable bug report.

I can empathize with being new to Jekyll and not understanding all of the components, but lets try to keep the scope of this issue narrow enough that there is a clear action that needs to be taken. Otherwise, there is always https://talk.jekyllrb.com

@swizca

This comment has been minimized.

Show comment
Hide comment
@swizca

swizca Jan 26, 2017

@pathawks The issue has not changed - please do not mistake it. The OP stands, and stands alone in and of and by itself.

Please don't munge it. DirtyF tried (with some success) to help in the mean time. tomjohnson1492 asked for some feedback, which I gave. Do not mistake the discussion for the issue, its status, or the OP.

swizca commented Jan 26, 2017

@pathawks The issue has not changed - please do not mistake it. The OP stands, and stands alone in and of and by itself.

Please don't munge it. DirtyF tried (with some success) to help in the mean time. tomjohnson1492 asked for some feedback, which I gave. Do not mistake the discussion for the issue, its status, or the OP.

@tomjoht

This comment has been minimized.

Show comment
Hide comment
@tomjoht

tomjoht Jan 26, 2017

Contributor

I'm just gathering a bit more information on this topic. I think the documentation should clarify the order of interpretation/processing. swizca's response enlarges the scope with more general issues and more broad debugging, but I'll keep this doc narrowly focused. give me a day or so and I'll submit a draft.

one surprise to me is with code block conversions to html and syntax highlighting. I thought this was the Markdown filter's job, but that's not the case. For example:

{% highlight javascript %}
console.log('alert');
{% endhighlight %}

this renders into HTML without involving the Markdown filter. The highlight tag is also not part of Liquid either. This is being transformed through my rouge gem, right? Does that processing happen before Liquid?

On the other hand, if I have this:

```javascript
console.log('alert');
```

Then the conversion happens via the Markdown filter, right?

I get a little confused about what is Liquid and what isn't. Jekyll defines some custom filters or tags that aren't part of Liquid but rather part of Jekyll and or one of its gems.

Contributor

tomjoht commented Jan 26, 2017

I'm just gathering a bit more information on this topic. I think the documentation should clarify the order of interpretation/processing. swizca's response enlarges the scope with more general issues and more broad debugging, but I'll keep this doc narrowly focused. give me a day or so and I'll submit a draft.

one surprise to me is with code block conversions to html and syntax highlighting. I thought this was the Markdown filter's job, but that's not the case. For example:

{% highlight javascript %}
console.log('alert');
{% endhighlight %}

this renders into HTML without involving the Markdown filter. The highlight tag is also not part of Liquid either. This is being transformed through my rouge gem, right? Does that processing happen before Liquid?

On the other hand, if I have this:

```javascript
console.log('alert');
```

Then the conversion happens via the Markdown filter, right?

I get a little confused about what is Liquid and what isn't. Jekyll defines some custom filters or tags that aren't part of Liquid but rather part of Jekyll and or one of its gems.

@pathawks

This comment has been minimized.

Show comment
Hide comment
@pathawks

pathawks Jan 26, 2017

Member

This is being transformed through my rouge gem, right?

{% highlight %} is a Liquid tag, provided by Jekyll. It is rendered by Liquid, which passes the content to Jekyll, which uses Pygments or Rouge to add syntax highlighting. The result is HTML.

We have been trying to standardize syntax highlighting, and I believe that in Kramdown, the Markdown style syntax highlighting will be parsed by Kramdown, passed to Jekyll, and then Jekyll will treat it the same as the highlight Liquid tag.

Member

pathawks commented Jan 26, 2017

This is being transformed through my rouge gem, right?

{% highlight %} is a Liquid tag, provided by Jekyll. It is rendered by Liquid, which passes the content to Jekyll, which uses Pygments or Rouge to add syntax highlighting. The result is HTML.

We have been trying to standardize syntax highlighting, and I believe that in Kramdown, the Markdown style syntax highlighting will be parsed by Kramdown, passed to Jekyll, and then Jekyll will treat it the same as the highlight Liquid tag.

@tomjoht

This comment has been minimized.

Show comment
Hide comment
@tomjoht

tomjoht Jan 27, 2017

Contributor

Thanks @pathawks. So even though Shopify's Liquid doesn't mention highlight as a tag anywhere in their docs, this is in fact Liquid after all? I don't quite understand that. Liquid is more than just what is in Shopify's docs?

Contributor

tomjoht commented Jan 27, 2017

Thanks @pathawks. So even though Shopify's Liquid doesn't mention highlight as a tag anywhere in their docs, this is in fact Liquid after all? I don't quite understand that. Liquid is more than just what is in Shopify's docs?

@pathawks

This comment has been minimized.

Show comment
Hide comment
@pathawks

pathawks Jan 27, 2017

Member

Any Gem can add Liquid tags, yes. It is extensible.

Member

pathawks commented Jan 27, 2017

Any Gem can add Liquid tags, yes. It is extensible.

@ashmaroli

This comment has been minimized.

Show comment
Hide comment
@ashmaroli

ashmaroli Jan 27, 2017

Member

So even though Shopify's Liquid doesn't mention highlight as a tag anywhere in their docs, this is in fact Liquid after all? I don't quite understand that. Liquid is more than just what is in Shopify's docs?

@tomjohnson1492 In line with Pat's reply to above, Shopify has two sets of documentation for their Liquid language. One for Liquid Core and another for use within shopify.com templates.

The filters and tags mentioned in the Core Documentation will be available for use with Jekyll (some filters require v4.0 circa Jekyll v3.5 though)
The extra filters and tags you'll find at shopify.com is only available for their shops.

Apart from the highlight tag used in Jekyll Core, Official Jekyll plugins add a few more. Notably, {% seo %}, {% css -- %}, {% gist -- %} etc, etc.

Its not just Liquid that's extensible. Jekyll itself is. One can use plugins to modify Jekyll by adding tags/filters/commands/subcommands/whatever.. the only catch being that it wont be supported by Github Pages which in itself is a Jekyll-modifying-plugin.

Member

ashmaroli commented Jan 27, 2017

So even though Shopify's Liquid doesn't mention highlight as a tag anywhere in their docs, this is in fact Liquid after all? I don't quite understand that. Liquid is more than just what is in Shopify's docs?

@tomjohnson1492 In line with Pat's reply to above, Shopify has two sets of documentation for their Liquid language. One for Liquid Core and another for use within shopify.com templates.

The filters and tags mentioned in the Core Documentation will be available for use with Jekyll (some filters require v4.0 circa Jekyll v3.5 though)
The extra filters and tags you'll find at shopify.com is only available for their shops.

Apart from the highlight tag used in Jekyll Core, Official Jekyll plugins add a few more. Notably, {% seo %}, {% css -- %}, {% gist -- %} etc, etc.

Its not just Liquid that's extensible. Jekyll itself is. One can use plugins to modify Jekyll by adding tags/filters/commands/subcommands/whatever.. the only catch being that it wont be supported by Github Pages which in itself is a Jekyll-modifying-plugin.

@tomjoht

This comment has been minimized.

Show comment
Hide comment
@tomjoht

tomjoht Jan 27, 2017

Contributor

Thanks for the explanation. This link -- Variations of Liquid -- also helped.

Contributor

tomjoht commented Jan 27, 2017

Thanks for the explanation. This link -- Variations of Liquid -- also helped.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment