Add template inheritance spec #75

jazzdan · 2014-04-14T21:07:39Z

This adds a specification for hogan.js style template inheritance. As of time of writing, three mustache implementations are 100% compatible with this implementation:

Here's how it works:

{{! header.mustache }}
<head>
  <title>{{$title}}Default title{{/title}}</title>
</head>

{{! base.mustache }}
<html>
  {{$header}}{{/header}}
  {{$content}}{{/content}}
</html>

{{! mypage.mustache }}
{{<base}}
  {{$header}}
    {{<header}}
      {{$title}}My page title{{/title}}
    {{/header}}
  {{/header}}

  {{$content}}
    <h1>Hello world</h1>
  {{/content}}
{{/base}}

Rendering mypage.mustache would output:

<html><head><title>My page title</title></head><h1>Hello world</h1></html>

A full specification can be seen below.

At @etsy, much like @twitter (see #38), we often want to have one template where we maintain the "bones" of the page and each real page on the site merely replaces sections of that page with appropriate content. A simple example of this can be seen above, where a page can grab base, getting all of our analytics scripts and other markup that every page needs, while retaining the ability to swap in a completely different header. Alternatives, such as copying the bones to every top level template, or simulating this behavior with a multi-step mustache render on the server, are not ideal.

Etsy is in the process of rolling out the new version of mustache.php that supports template inheritance to our production cluster after testing this implementation extensively. Hogan.js and mustache.java are both being used in production at Twitter.

Many thanks to @dtb, @mdg, @bobthecow for their assistance. Special thanks to @spullara for quickly making mustache.java compatible with this spec.

bobthecow · 2014-04-14T21:18:35Z

Hey, @groue — I remember you saying GRMustache was moving to this style of inheritance as well... Do you mind running this spec against it and seeing how it works?

groue · 2014-04-15T05:58:17Z

Hi @bobthecow. Yes, GRMustache passes all those tests (modulo white-space conformance, that GRMustache has never focused on).

Generally speaking, GRMustache passes all inheritance tests from this PR, from hogan.js, and spullara/mustache.java.

I'm curious about your own implementations' conformance to GRMustache tests as well - they cover a few cases not included in this PR:

the fact that partial extraction does not mess with template inheritance. The goal is to make sure one can always extract a partial from a template without worrying about breaking inheritance: https://github.com/groue/GRMustache/blob/GRMustache7/src/tests/Public/v7.0/Suites/groue:GRMustache/GRMustacheSuites/inheritable_partials.json (many tests)
the fact that when there are several conflicting inheritable sections, the last one wins: https://github.com/groue/GRMustache/blob/GRMustache7/src/tests/Public/v7.0/Suites/groue:GRMustache/GRMustacheSuites/inheritable_partials.json#L72-L96
the fact that inheritable section names are totally independent from view-model keys is important to clearly state as well: https://github.com/groue/GRMustache/blob/GRMustache7/src/tests/Public/v7.0/Suites/groue:GRMustache/GRMustacheSuites/inheritable_sections.json

groue · 2014-04-15T06:42:58Z

@jazzdan wrote:

As of time of writing, three mustache implementations are 100% compatible with this implementation:

There are four or them, actually :-) You can add groue/GRMustache.

bobthecow · 2014-04-15T06:46:33Z

Your third bullet (separate namespace for inheritance vs. regular variable keys) was addressed in the PHP implementation.

bobthecow · 2014-04-15T07:04:54Z

@groue This one is interesting, and I'm not sure I'm convinced it's the right way to treat it:

https://github.com/groue/GRMustache/blob/GRMustache7/src/tests/Public/v7.0/Suites/groue:GRMustache/GRMustacheSuites/inheritable_partials.json#L59-L64

Sell me on it :)

bobthecow · 2014-04-15T07:05:31Z

I just tested the PHP implementation against your second bullet point (several conflicting inheritable sections), and that works as well.

bobthecow · 2014-04-15T07:18:53Z

Okay. Your inheritable partials tests mostly pass in the PHP implementation (2 failures, 5 errors). The two failures are due to treatment of blocks inside sections (my "not convinced" comment above).

The five errors are exceptions thrown because Mustache.php, like Hogan.js, throws errors for things like this:

{{< parent }}
  {{# section }}this will throw an exception!{{/ section }}
{{/ parent }}

groue · 2014-04-15T07:54:57Z

Thanks for your time @bobthecow :-)

https://github.com/groue/GRMustache/blob/GRMustache7/src/tests/Public/v7.0/Suites/groue:GRMustache/GRMustacheSuites/inheritable_partials.json#L59-L64

Sell me on it :)

Generally speaking, I tend to think that this behavior is required by those two arguments:

The content of inheritable sections is rendered in the current context: jazzdan@ab22750#diff-d3a1efe06cc23358b500e2bdab06504eR22
the rendering of inheritable sections is triggered by the layout. It may be overriden or not. The what is decided by inherited templates, but the when is decided by the layout. This "moment" may be inside a regular mustache section, with some particular context.

The two arguments join on the "context" word: if the layout template changes the context, inherited sections need to render in this context. Hence the test you question, and the example below:

layout.mustache

{{!--
Layout for a document.

Exposed inheritable sections:
- $title: What to be rendered for the document title.
          When rendered, the `document` object is at the top of the context stack.
          Default value: `document.title`.
}}
{{#document}}
    {{$title}}
        {{title}}
    {{/title}}
    ...
{{/document}}

essay.mustache

{{<layout}}
    {{$title}}
        {{! Prepend "Essay" to the document title }}
        Essay: {{title}}
    {{/title}}
{{/layout}}

Rendering:

render('essay', { document: ...});

groue · 2014-04-15T07:57:48Z

The five errors are exceptions thrown because Mustache.php, like Hogan.js, throws errors for things like this:
{{< parent }}
  {{# section }}this will throw an exception!{{/ section }}
{{/ parent }}

I see. What is the intent? To prevent users to embed ignored content? If so, do you throw exception for plain text as well? If not, what is the intent of you exception?

{{< parent }}
  Not only sections, but this text also is ignored.
{{/ parent }}

bobthecow · 2014-04-15T14:38:24Z

@groue I'm actually leaning toward that (see comment). The reason it came up was things like this:

{{< parent }}
  {{# items }}
    {{$ block }}wheee!{{/ block }}
  {{/ items }}
{{/ parent }}

… which are weird. I'd be in favor of explicitly whitelisting only a few things that can live in the top level of a parent tag:

Whitespace
{{! comments }}
{{$ blocks }}
{{=[[ ]]=}}

groue · 2014-04-15T15:18:03Z

@bobthecow All right, I understand, and I'm not against returning errors for all ignored tags, but the ones you are listing: white space, comments, inheritable sections, and change-delimiter tags.

BTW, GRMustache also has tests for such parsing errors: https://github.com/groue/GRMustache/blob/GRMustache7/src/tests/Public/v7.0/Suites/groue:GRMustache/GRMustacheSuites/tag_parsing_errors.json & https://github.com/groue/GRMustache/blob/GRMustache7/src/tests/Public/v7.0/Suites/groue:GRMustache/GRMustacheSuites/expression_parsing_errors.json

Back on your exceptions:

There are only six GRMustache inheritance tests which involve regular sections:

Only 1 of them embeds a regular section inside a inheritable section, and has your implementation throw an exception. Let's forget about this one:

https://github.com/groue/GRMustache/blob/GRMustache7/src/tests/Public/v7.0/Suites/groue:GRMustache/GRMustacheSuites/inheritable_partials.json#L158-164

The others should not raise any exception. I mean, your implementation does not have to handle them, but you may be interested in looking at them more precisely:

https://github.com/groue/GRMustache/blob/GRMustache7/src/tests/Public/v7.0/Suites/groue:GRMustache/GRMustacheSuites/inheritable_partials.json#L18-23

Rationale: if a given string is a valid template, then any substring that leaves open/close tags intact can be extracted to a partial, or an inheritable partial. Why? Because one should be able to extract a partial without any worry, as long as the resulting templates are syntactically correct. Application: {{#items}}{{.}}{{/items}} can be turned into {{#items}}{{>partial}}{{/items}}, or {{#items}}{{<partial}}...{{/partial}}{{/items}}.
https://github.com/groue/GRMustache/blob/GRMustache7/src/tests/Public/v7.0/Suites/groue:GRMustache/GRMustacheSuites/inheritable_partials.json#L58-L64

This one has been explained above
https://github.com/groue/GRMustache/blob/GRMustache7/src/tests/Public/v7.0/Suites/groue:GRMustache/GRMustacheSuites/inheritable_partials.json#L151-157

This one is somewhat superseeded now that separate namespace for inheritance vs. regular variable keys is tested in some other place.
https://github.com/groue/GRMustache/blob/GRMustache7/src/tests/Public/v7.0/Suites/groue:GRMustache/GRMustacheSuites/inheritable_partials.json#L165-174

This one comes from an spullara/mustache.java issue
https://github.com/groue/GRMustache/blob/GRMustache7/src/tests/Public/v7.0/Suites/groue:GRMustache/GRMustacheSuites/inheritable_partials.json#L175-181

A contrieved recursion example

bobthecow · 2014-04-15T15:33:19Z

Those aren't the ones that raised exceptions, these are:

[
  // ...
  {
    "name": "The content of the last inheritable sections in partials is rendered in the inherited section",
    "data": { },
    "template": "{{<partial1}}{{$inheritable}}1{{/inheritable}}{{>partial2}}{{$inheritable}}4{{/inheritable}}{{/partial1}}",
    "partials": {
      "partial1": "{{$inheritable}}ignored{{/inheritable}}",
      "partial2": "{{$inheritable}}2{{/inheritable}}{{$inheritable}}3{{/inheritable}}" },
    "expected": "4"
  },
  {
    "name": "The content of the last inheritable sections in partials is rendered in the inherited section",
    "data": { },
    "template": "{{<partial1}}{{$inheritable}}1{{/inheritable}}{{>partial2}}{{/partial1}}",
    "partials": {
      "partial1": "{{$inheritable}}ignored{{/inheritable}}",
      "partial2": "{{$inheritable}}2{{/inheritable}}{{$inheritable}}3{{/inheritable}}" },
    "expected": "3"
  },
  // ...
  {
    "name": "Partials in inheritable partials can override inheritable sections",
    "data": { },
    "template": "{{<partial2}}{{>partial1}}{{/partial2}}",
    "partials": {
        "partial1": "{{$inheritable}}partial1{{/inheritable}}",
        "partial2": "{{$inheritable}}ignored{{/inheritable}}" },
    "expected": "partial1"
  },
  // ...
  {
    "name": "Templates sections can not override inheritable sections in inheritable partial",
    "data": { "section": true },
    "template": "{{<partial}}{{#section}}inherited{{/section}}{{/partial}}",
    "partials": { "partial": "{{$section}}success{{/section}}" },
    "expected": "success"
  },
  // ...
]

Note that they all include tags besides inheritable sections inside parent tags, which is the current criteria for throwing an exception.

I did just realize that two of these revolve around allowing partials to define inheritable sections. I'm inclined to push back against this one and say only inheritable sections actually defined inside the parent tag count.

groue · 2014-04-15T15:37:21Z

All right, you can do as you wish. My take on that point is as stated above:

if a given string is a valid template, then any substring that leaves open/close tags intact can be extracted to a partial, or an inheritable partial. Why? Because one should be able to extract a partial without any worry, as long as the resulting templates are syntactically correct.

bobthecow · 2014-04-15T15:38:40Z

All right, you can do as you wish.

That's not constructive. I'm trying to have a conversation and come to a consensus about common functionality so that we can actually standardize this behavior.

bobthecow · 2014-04-15T15:41:39Z

All right, you can do as you wish. My take on that point is as stated above:

if a given string is a valid template, then any substring that leaves open/close tags intact can be extracted to a partial, or an inheritable partial. Why? Because one should be able to extract a partial without any worry, as long as the resulting templates are syntactically correct.

But that's currently not the case. Consider this template:

{{=<% %>=}}
<% win %>
{{ fail }}

You can't extract any one line of this template into a partial, because the whole thing will break.

I'm just saying that I feel like inheritable sections are a special case too.

groue · 2014-04-15T15:59:41Z

That's not constructive. I'm trying to have a conversation and come to a consensus about common functionality so that we can actually standardize this behavior.

I'm sorry. I did not want to pollute @jazzdan's PR. I did state my reason after. Your answer:

But that's currently not the case.

Yes, you are right. Set delimiter tags ruin everything. I wish they were not there. I never use them. They are a hack. That's why I think my take is still valid. And this is why GRMustache tests for snippet-extraction-into-partials at several levels. So that I'm sure my users can extract whatever they want into a partial, as long as it looks sensible to them. Who am I to say it's forbidden? Not implemented, why not. But forbidden? When they have a legitimate need? When it can be implemented in a rigourous way? When it has been implemented?

You may not include those tests in this spec. This is what I meant by "you can do as you wish". But I won't remove this feature from GRMustache.

groue · 2014-04-15T16:04:09Z

About free partials extraction, let me quote @defunkt on http://mustache.github.io/mustache.5.html

In this way you may want to think of partials as includes, or template expansion, even though it's not literally true.

For example, this template and partial:
base.mustache:
<h2>Names</h2>
{{#names}}
  {{> user}}
{{/names}}

user.mustache:
<strong>{{name}}</strong>
Can be thought of as a single, expanded template:
<h2>Names</h2>
{{#names}}
  <strong>{{name}}</strong>
{{/names}}

This is the spirit.

bobthecow · 2014-04-15T16:57:11Z

Allowing partials inside parent tags changes what would be a parse error ("unexpected section tag inside a parent tag!") into a runtime exception, because you can't be sure that a partial won't include something that isn't allowed directly inside a parent tag. It complicates things :)

groue · 2014-04-15T17:09:29Z

Allowing partials inside parent tags changes what would be a parse error ("unexpected section tag inside a parent tag!") into a runtime exception, because you can't be sure that a partial won't include something that isn't allowed directly inside a parent tag. It complicates things :)

What runtime exception are you talking about? I'm not aware of runtime errors I should raise when rendering plain Mustache templates.

In GRMustache, the only runtime errors are for missing filters (a feature added by GRMustache, not in the spec: unknown filter f in {{ f(x) }}), and inconsistent mixing of HTML and text, a pretty rare programming error that can happen in GRMustache only (text templates are not in the spec: https://github.com/groue/GRMustache/blob/master/Guides/html_vs_text.md).

bobthecow · 2014-04-15T17:15:17Z

If things like section tags are not allowed directly inside parent tags, this would be a parse error:

{{< parent }}
  {{# foo }}...{{/ foo }}
{{/ parent }}

But this would be a runtime error:

{{< parent }}
  {{> partial }}
{{/ parent }}

partial.mustache

{{# foo }}...{{/ foo }}

groue · 2014-04-15T17:21:38Z

Well, in your last example, we would have a parse error because the {{> partial }} tag would be disallowed inside the {{< parent }}...{{/}} tag. So we still have no runtime error here.

bobthecow · 2014-04-15T17:24:49Z

This thing you just said:

Well, in your last example, we would have a parse error because the {{> partial }} tag would be disallowed inside the {{< parent }}...{{/}} tag. So we still have no runtime error here.

Directly contradicts that other thing you keep saying:

if a given string is a valid template, then any substring that leaves open/close tags intact can be extracted to a partial, or an inheritable partial.

… and, in fact, agrees completely with what I'm trying to say. So maybe we misunderstand each other :)

groue · 2014-04-15T17:35:06Z

@bobthecow, I think you mix two questions:

how are processed tags inside a inherited template (the famous white-list of white-space, change delimiters, comment, inheritable section)
the question raised by the GRMustache tests about partial extraction.

The two topics are different.

Your last example talks about the white-list. Since partial tags should raise a parsing exception, according to this white-list rule, this example would not raise any runtime exception.

About the second topic: please consider that all the partial-extraction tests are rendered by GRMustache. They work. They have a meaning. They test the fact that one can extract a partial from any template snippet, as long as the template snippet can be rendered, and that the two new templates are valid.

bobthecow · 2014-04-15T17:37:39Z

I was, and have been, talking about the four tests with exceptions I cited above. They throw exceptions because they include tags that are not on the whitelist. I thought you were talking about them as well, and saying that they should be valid.

groue · 2014-04-15T17:45:52Z

The three first tests are about choosing which overriding section wins, when several ones could be rendered. They make sure the last one wins. They do not involve the white-list.

The last test could raise an exception, with the white-list. Today, it just makes sure an regular section is ignored, and does not mess with inheritance.

bobthecow · 2014-04-15T17:48:50Z

… partial tags should raise a parsing exception, according to this white-list rule…

All three of these raise exceptions, because they all have partials tags directly inside parent tags:

{{<partial1}}
  {{$inheritable}}1{{/inheritable}}
  {{>partial2}}
  {{$inheritable}}4{{/inheritable}}
{{/partial1}}

{{<partial1}}
  {{$inheritable}}1{{/inheritable}}
  {{>partial2}}
{{/partial1}}

{{<partial2}}
  {{>partial1}}
{{/partial2}}

groue · 2014-04-15T17:49:44Z

OMG let me check my tests

groue · 2014-04-15T17:55:00Z

Ha yeah, I got it :-)

Those tests are a consequence of general partial extraction. Given the following template:

template.mustache
{{< parent }}
    {{$ inheritable}}...{{/ inheritable }}
{{/ parent }}

The user may want to extract the content into a partial:

template.mustache
{{< parent }}
    {{> partial }}
{{/ parent }}

partial.mustache
{{$ inheritable}}...{{/ inheritable }}

So if I come back to the white-list, I would state the partial tags should be white-listed as well.

groue · 2014-04-15T17:58:53Z

The last example may look contrived. I have to imagine somebody who wants to reuse common inheritable sections for various layouts, and then extracts them into a partial.

The specs come from the semi-official specification mustache/spec#75

The implementation follows the spec proposal: mustache/spec#75 which is supported by at least the following Mustache implementations: - hogan.js - mustache.php - mustache.java - GRMustache (Obj-C) and GRMustache.swift (Swift) - Text::Caml (Perl) - hxmustache (Haxe) - MuttonChop (Swift)

The specs come from the semi-official specification mustache/spec#75

The implementation follows the spec proposal: mustache/spec#75 which is supported by at least the following Mustache implementations: - hogan.js - mustache.php - mustache.java - GRMustache (Obj-C) and GRMustache.swift (Swift) - Text::Caml (Perl) - hxmustache (Haxe) - MuttonChop (Swift)

The specs come from the semi-official specification mustache/spec#75

jgonggrijp

This is a really old PR and I'm a bit of an outsider since I only just wrote my own Mustache implementation, but I figured a review might still be useful since this PR hasn't been merged yet and I might offer a "fresh perspective" as a newcomer. For clarity, my own implementation doesn't support this spec yet, but I do want to support it.

Semantically and syntactically speaking, I'm perfectly happy with the spec as-is, and I agree that it should be merged in v2. The fact that it is already existing practice in multiple implementations also makes the case for this spec stronger in my opinion. I would however propose to make two superficial (but important) changes:

Change the spec name from inheritance.yml to ~inheritance.yml to mark it as an optional module. It complicates the implementation a bit, so it might deter new implementers otherwise. Edit to add: after Add template inheritance spec #75 (comment), I would suggest ~parametric-partial.yml instead.
Make the jargon much more consistent. I found {{$block}} being referred to as "block", "parameter" and "substitution" and {{<parent}} as "parent", "super", "include" and even "partial". ~~Especially "partial" should really be avoided in my opinion, as it is needlessly confusing.~~ I started to make change suggestions for the incorrect appearances of "partial", but I didn't go all the way through. I would suggest using ~~only "parent" or "super" for {{<parent}} and~~ "parameter block" for {{$block}} when used by itself or "argument block" when used inside {{<parent}}...{{/parent}}.. Edit to add: I changed my opinion about this and I now believe that "parent", "super" and "inheritance" are actually the words that should be avoided. See Add template inheritance spec #75 (comment).

Some other remarks/observations:

I'm in favor of specifying that interpolation and section tags inside argument blocks should be evaluated in the context of the parent template, as proposed by @gasche in Add template inheritance spec #75 (comment) after finding that this is the existing practice in mustache.php. This makes it possible to implement inheritance by inlining the parent template, making precompiled templates potentially faster. It is also more consistent with the same paths already occurring inside the parameter block default in the parent template.
I really like the chosen sigils/indicators/whathever they are called. < makes sense because in a way, it is dual to the > for partials. $ makes sense because the character is used for variable names or parameter expansions in several programming languages.
Maybe it would be an option to open a "future" branch so that changes for v2 can already be merged into the repository?

jgonggrijp · 2021-03-09T17:08:15Z

specs/inheritance.yml

+  - name: Overridden partial
+    desc: Overridden partial
+    data: { }
+    template: "test {{<partial}}{{$stuff}}override{{/stuff}}{{/partial}}"
+    partials:
+      partial: "{{$stuff}}...{{/stuff}}"
+    expected: "test override"


I think the wording "partial" is rather unfortunate here, as neither the parent nor the template involves {{>partial}} syntax. I suggest the following instead.

Suggested change

- name: Overridden partial

desc: Overridden partial

data: { }

template: "test {{<partial}}{{$stuff}}override{{/stuff}}{{/partial}}"

partials:

partial: "{{$stuff}}...{{/stuff}}"

expected: "test override"

- name: Overriden content with text in front

desc: Overriden content with text in front of parent

data: { }

template: "test {{<parent}}{{$stuff}}override{{/stuff}}{{/parent}}"

partials:

parent: "{{$stuff}}...{{/stuff}}"

expected: "test override"

I retract the above comment.

jgonggrijp · 2021-03-09T17:10:49Z

specs/inheritance.yml

+  - name: Two overridden partials
+    desc: Two overridden partials with different content


Suggested change

- name: Two overridden partials

desc: Two overridden partials with different content

- name: Overrides in twice inherited parent

desc: Overrides in twice inherited parent with different content

I retract the above comment.

jgonggrijp · 2021-03-09T17:12:57Z

specs/inheritance.yml

+    expected: "peaked\n\n:(\n"
+
+  - name: Inherit indentation
+    desc: Inherit indentation when overriding a partial


Suggested change

desc: Inherit indentation when overriding a partial

desc: Inherit indentation when overriding a parent

I retract the above comment.

jgonggrijp · 2021-03-09T17:17:38Z

specs/inheritance.yml

+  - name: Override partial with newlines
+    desc: Override partial with newlines


Suggested change

- name: Override partial with newlines

desc: Override partial with newlines

- name: Override parent with newlines

desc: Override parent with newlines

I retract the above comment.

jgonggrijp · 2021-03-09T17:42:08Z

specs/inheritance.yml

+      stop:
+          hammer time
+
+    - name: Only one override


Suggested change

- name: Only one override

- name: Only one override

gasche · 2021-03-09T21:49:09Z

Thanks for the carefully worded message! I agree that it's nice to push this further: I think that with the amount of implementations that support the feature, it's a bit silly to refer to it as a PR to the original spec.

Semantically and syntactically speaking, I'm perfectly happy with the spec as-is, and I agree that it should be merged in v2. [...]

Change the spec name from inheritance.yml to ~inheritance.yml to mark it as an optional module. It complicates the implementation a bit, so it might deter new implementers otherwise.

If this is made an optional part of the spec, does it need to correspond to a major-version bump for the specification? I haven't thought hard about the versioning scheme but I would think as adding optional aspects as a backward-compatibility-preserving changes (implementations remain compliant). (One exception would be an implementation that provides a feature using the same syntax as proposed here but with different semantics, then I guess it would become non-compliant?)

Make the jargon much more consistent. I found {{$block}} being referred to as "block", "parameter" and "substitution" and {{<parent}} as "parent", "super", "include" and even "partial". Especially "partial" should really be avoided in my opinion, as it is needlessly confusing. I started to make change suggestions for the incorrect appearances of "partial", but I didn't go all the way through. I would suggest using only "parent" or "super" for {{<parent}} and "parameter block" for {{$block}} when used by itself or "argument block" when used inside {{<parent}}...{{/parent}}..

I agree that the terminology could be made more consistent. I myself use parameter block for {{$foo}}...{{/foo}} (the "block" part is important because it's not just a tag, it can contain an arbitrary sub-template), and sometimes just "parameter" when the context makes it very clear. I think that "block" means generically anything with an opening and a closing form (usually {{/foo}}) and stuff in the middle.

Regarding the whole feature or {{<foo}}, I personally use the wording "partial with parameters": it behaves exactly like partials {{>foo}} (it expands to a sub-template), except it can take parameters. This is a wording that makes sense to my community (more oriented towards functional programming than oo-style inheritance), but then it is not perfect either because the shadowing semantics of partial blocks are unintuitive if you think of them as function calls (the outermost definition wins, not the innermost one as one would expect). I guess I mean that I haven't found a perfect name for this part of the feature, and we should be prepared for different terminologies to survive.

Another remark: upon implementing the feature I found that it shows the limits of the current way whitespace is handled in Mustache (the notion of standalone tags, and the indentation of partials). Ideally the user would want:

{{$foo}}...{{/foo}} to count as a single "standalone tag" if it is surrounded by whitespace
the indentation of the parameter block to be substracted at rendering time, instead of being added like the indentation of a partial (with or without parameters)
the way we count indentation to be slightly different when {{$foo}} is on its own line and the comment following is indented.

Long story short, I went down a rabbit hole of finding better specifications for whitespace handling in Mustache, and they tended to be more complex, more natural for the user, and incompatible with existing implementations / the spec. in corner cases. I gave up and went back to something mostly-standard (my implementation does substract the indentation of the parameter block, iirc.), but it remains a (minor) sticking point with the feature and its specification.

jgonggrijp · 2021-03-10T03:36:33Z

Good point about "partials without parameters" and "partials with parameters". I now realize they are more similar than I thought, and also that the implementation can probably be more similar than I thought.

If this is made an optional part of the spec, does it need to correspond to a major-version bump for the specification? I haven't thought hard about the versioning scheme but I would think as adding optional aspects as a backward-compatibility-preserving changes (implementations remain compliant). (One exception would be an implementation that provides a feature using the same syntax as proposed here but with different semantics, then I guess it would become non-compliant?)

I was thinking along these lines, too, but I didn't dare write it down. I'm glad you did. With it being an optional module, I feel it could be a minor version bump.

jgonggrijp · 2021-03-15T12:18:01Z

Since my review and previous comment, I have radically changed my opinion about jargon consistency. I now believe that "parent", "super" and "inheritance" are the needlessly confusing words that should be avoided. The thing we're talking about here is just a parametric partial. I would suggest renaming the spec to ~parametric-partial.yml. Thanks to @gasche for making me realize this.

jgonggrijp · 2021-03-15T13:02:48Z

@gasche

Another remark: upon implementing the feature I found that it shows the limits of the current way whitespace is handled in Mustache (the notion of standalone tags, and the indentation of partials). Ideally the user would want:

{{$foo}}...{{/foo}} to count as a single "standalone tag" if it is surrounded by whitespace

Could you elaborate on this? The way I've understood standalone tags so far, it just means that a single tag that is all alone on a line shouldn't leave a blank line in the output, i.e.,

{{!comment}}
foo
{{#section}}
    content
{{/section}}

should produce

foo
    content

(assuming that section is truthy) rather than


foo

    content

. Given this interpretation, I have no imagination of what it would even mean for a parameter block as a whole to count as a standalone tag. Consider the following example:

foo
    {{$parameter}}content{{/parameter}}
bar

What do you believe should happen? Surely not that the entire line containing content should be removed from the output?

the indentation of the parameter block to be substracted at rendering time, instead of being added like the indentation of a partial (with or without parameters)

the way we count indentation to be slightly different when {{$foo}} is on its own line and the comment following is indented.

If you mean that

{{!partial.mustache}}
foo
    bar
    {{$parameter1}}baz{{/parameter1}}
    {{$parameter2}}foobar{{/parameter2}}

{{!template.mustache}}
foobaz
    barbaz
    {{<partial}}
        foofoo
            {{$parameter1}}barbar{{/parameter1}}
      {{$parameter2}}
          bazbaz
      {{/parameter2}}
    {{/partial}}

should output

foobaz
    barbaz
    foo
        bar
        barbar
            bazbaz

then I think I agree with you.

Long story short, I went down a rabbit hole of finding better specifications for whitespace handling in Mustache, and they tended to be more complex, more natural for the user, and incompatible with existing implementations / the spec. in corner cases. I gave up and went back to something mostly-standard (my implementation does substract the indentation of the parameter block, iirc.), but it remains a (minor) sticking point with the feature and its specification.

Does this really need to be fully specified before merging, though? I think the existing spec is worthwhile as-is, even if it leaves some freedom to implementers with regard to whitespace handling.

gasche · 2021-03-19T07:41:22Z

Regarding the terminology: I would be a bit careful here, to accommodate several viewpoints. I think that to some people thinking of those new partials as "functions with parameters" will be most helpful, but some other would be familiar with the "object-oriented inheritance" view. We could have a description that mentions both views, and I think we could manage to consistently/clearly use two terminologies.

Note that the two notions are not completely equivalent in what semantics they suggest: in case of shadowing (two nested partial-with-parameters define parameters of the same name), the function view would suggest that the innermost definition (the callee) should override the outermost one, while the inheritance view suggests that the outermost definition (the parent class) should override the innermost one; and it is that latter view that was chosen (possibly without realizing the choice) in the original implementation of partial-with-parameters, and followed by further implementations.

gasche · 2021-03-19T08:00:41Z

Regarding whitespace: I think that, ideally

  {{<foo}}{{$param}}bar{{/param}}{{/foo}}

and

  {{<foo}}
    {{$param}}bar{{/param}}
  {{/foo}}

should result in the same printing. However, with the current notion of "standalone" tag, then {{<foo}} and {{/foo}} are standalone in the second example but not in the first one. This suggests that the second example should not include the newlines following {{>foo}} or {{/foo}}, but according to our current specification it probably should include the newline after {{/foo}} in the first example.

I think that a better concept of whitespace handling should be about whether templating constructs (at rendering time) emit "visible" content on the current line, or they don't. If a "standalone" construct (not {{foo}}, which counts as visible even when it emits whitespace only) is on a line without any visible content, then the whitespace before it is treated as indentation and the whitespace after it is elided from the output. With this definition, the use of parameter blocks at definition site is clearly not visible, and so both examples should elide the newline. But (1) this is a significant change compared to the current definition and in some cases it changes behavior (for example it makes {{#foo}}{{#bar}} act as a standalone sequence of tokens, which is nice but incompatible), and (2) I don't know how difficult it would be to implement for other Mustache implementations (I implemented this in a prototype as an experiment).

Does this really need to be fully specified before merging, though? I think the existing spec is worthwhile as-is, even if it leaves some freedom to implementers with regard to whitespace handling.

No, I think that this does not need to be fully specified before merging, and adding that requirement could be fatal to this PR who already had a long and unsatisfying life. My personal opinion is that this PR should be merged (marked optional) into the specification, to recognize the current status that it is consensual among implementations, and largely implemented. It is a de-facto (optional) standard, and everyone's life would be easier if it was recognized as such.

My current understanding is that the notion of "standalone token" as currently somewhat-specified (you have to read a lot between the lines already) is not such a good fit for "we want the nice whitespace behavior a human would expect"; it is doing okay for existing constructs (although arguably {{#foo}}{{#bar}} should already be treated as standalone, don't we think?), and slightly worse for this new construct which introduces these parameter blocks that are sometimes in invisible/non-rendered position. This is life, we cannot have all things perfect, and I think it should be discussed separately.

jgonggrijp · 2021-03-19T11:57:34Z

@gasche Thank you for taking the effort to have this interesting discussion with me.

Regarding terminology, point taken. I should mention that my opinion is not so strong that I believe it should block merging. I would like to see it be more consistent, but since the terminology is not going to change the syntactical or semantical content of the specs, we are going to get the same result in the end and I'll have the same work to do in my own implementation.

For academic interest, I'll mention that parametric partials behave exactly like functions with default arguments. Indeed, nested function calls with defaults along the chain will give the same behavior where the outermost caller wins. On the other hand, a template that calls multiple (parametric) partials behaves more like composition than like multiple inheritance, when looking from an OOP perspective. Partials might be deeply nested within sections from the "inheriting" template, giving it complete control to wrap and organize the partials however the author wants. In OOP, inheritance generally isn't that flexible but composition is.

Regarding whitespace, I think I actually prefer the whitespace handling as it is currently specified, although I agree that {{#section}}{{#nested}} should ideally be recognized as two standalone tags. I would agree to moving that discussion to a separate issue.

@Danappelxx @spullara I could prepare a continuation PR that renames the spec to start with a ~ (to mark it as optional) and that makes the terminology more consistent (without necessarily enforcing one particular jargon). Would you welcome that?

gasche · 2021-03-21T14:00:23Z

I could prepare a continuation PR that renames the spec to start with a ~ (to mark it as optional) and that makes the terminology more consistent (without necessarily enforcing one particular jargon).

Personally I think this would be a good idea. Thanks for your active contributions to the mustache spec!

Danappelxx · 2021-04-21T02:18:19Z

Closing in favor of #125, which looks like it will be merged soon - further comments should be added there. Thank you to everyone for facilitating a healthy discussion.

@gasche

CHANGES: * Remove the AST without locations: now all functions build an AST with locations; in particular, parsing always provide located error messages. To ease backward-compatibility, the smart constructors still use the same interface, using dummy locations by default, with a With_locations module for users who wish to explicitly provide locations. (@gasche, rgrinberg/ocaml-mustache#65) * Support for "template inheritance" (partials with parameters) `{{<foo}} {{$param1}}...{{/param1}} {{$param2}}...{{/param2}} {{/foo}` following the widely-implemented semi-official specification mustache/spec#75 (@gasche, 58) * Partials are now supported in the `mustache` command-line tool (@gasche, rgrinberg/ocaml-mustache#57) They are interpreted as template inclusion: "{{>foo/bar}}" will include "foo/bar.mustache", relative to the current working directory. * Improve error messages (@gasche, rgrinberg/ocaml-mustache#47, rgrinberg/ocaml-mustache#51, rgrinberg/ocaml-mustache#56) Note: the exceptions raised by Mustache have changed, this breaks compatibility for users that would catch and deconstruct existing exceptions. * Add `render_buf` to render templates directly to buffers (@gasche, rgrinberg/ocaml-mustache#48) * When a lookup fails in the current context, lookup in parents contexts. This should fix errors when using "{{#foo}}" for a scalar variable 'foo' to check that the variable exists. (@gasche, rgrinberg/ocaml-mustache#49)

@gasche

CHANGES: * Remove the AST without locations: now all functions build an AST with locations; in particular, parsing always provide located error messages. To ease backward-compatibility, the smart constructors still use the same interface, using dummy locations by default, with a With_locations module for users who wish to explicitly provide locations. (@gasche, rgrinberg/ocaml-mustache#65) * Support for "template inheritance" (partials with parameters) `{{<foo}} {{$param1}}...{{/param1}} {{$param2}}...{{/param2}} {{/foo}` following the widely-implemented semi-official specification mustache/spec#75 (@gasche, 58) * Partials are now supported in the `mustache` command-line tool (@gasche, rgrinberg/ocaml-mustache#57) They are interpreted as template inclusion: "{{>foo/bar}}" will include "foo/bar.mustache", relative to the current working directory. * Improve error messages (@gasche, rgrinberg/ocaml-mustache#47, rgrinberg/ocaml-mustache#51, rgrinberg/ocaml-mustache#56) Note: the exceptions raised by Mustache have changed, this breaks compatibility for users that would catch and deconstruct existing exceptions. * Add `render_buf` to render templates directly to buffers (@gasche, rgrinberg/ocaml-mustache#48) * When a lookup fails in the current context, lookup in parents contexts. This should fix errors when using "{{#foo}}" for a scalar variable 'foo' to check that the variable exists. (@gasche, rgrinberg/ocaml-mustache#49)

@gasche

CHANGES: * Remove the AST without locations: now all functions build an AST with locations; in particular, parsing always provide located error messages. To ease backward-compatibility, the smart constructors still use the same interface, using dummy locations by default, with a With_locations module for users who wish to explicitly provide locations. (@gasche, rgrinberg/ocaml-mustache#65) * Support for "template inheritance" (partials with parameters) `{{<foo}} {{$param1}}...{{/param1}} {{$param2}}...{{/param2}} {{/foo}` following the widely-implemented semi-official specification mustache/spec#75 (@gasche, 58) * Partials are now supported in the `mustache` command-line tool (@gasche, rgrinberg/ocaml-mustache#57) They are interpreted as template inclusion: "{{>foo/bar}}" will include "foo/bar.mustache", relative to the current working directory. * Improve error messages (@gasche, rgrinberg/ocaml-mustache#47, rgrinberg/ocaml-mustache#51, rgrinberg/ocaml-mustache#56) Note: the exceptions raised by Mustache have changed, this breaks compatibility for users that would catch and deconstruct existing exceptions. * Add `render_buf` to render templates directly to buffers (@gasche, rgrinberg/ocaml-mustache#48) * When a lookup fails in the current context, lookup in parents contexts. This should fix errors when using "{{#foo}}" for a scalar variable 'foo' to check that the variable exists. (@gasche, rgrinberg/ocaml-mustache#49)

@gasche

CHANGES: * Remove the AST without locations: now all functions build an AST with locations; in particular, parsing always provide located error messages. To ease backward-compatibility, the smart constructors still use the same interface, using dummy locations by default, with a With_locations module for users who wish to explicitly provide locations. (@gasche, rgrinberg/ocaml-mustache#65) * Support for "template inheritance" (partials with parameters) `{{<foo}} {{$param1}}...{{/param1}} {{$param2}}...{{/param2}} {{/foo}` following the widely-implemented semi-official specification mustache/spec#75 (@gasche, 58) * Partials are now supported in the `mustache` command-line tool (@gasche, rgrinberg/ocaml-mustache#57) They are interpreted as template inclusion: "{{>foo/bar}}" will include "foo/bar.mustache", relative to the current working directory. * Improve error messages (@gasche, rgrinberg/ocaml-mustache#47, rgrinberg/ocaml-mustache#51, rgrinberg/ocaml-mustache#56) Note: the exceptions raised by Mustache have changed, this breaks compatibility for users that would catch and deconstruct existing exceptions. * Add `render_buf` to render templates directly to buffers (@gasche, rgrinberg/ocaml-mustache#48) * When a lookup fails in the current context, lookup in parents contexts. This should fix errors when using "{{#foo}}" for a scalar variable 'foo' to check that the variable exists. (@gasche, rgrinberg/ocaml-mustache#49)

@gasche

CHANGES: * Remove the AST without locations: now all functions build an AST with locations; in particular, parsing always provide located error messages. To ease backward-compatibility, the smart constructors still use the same interface, using dummy locations by default, with a With_locations module for users who wish to explicitly provide locations. (@gasche, rgrinberg/ocaml-mustache#65) * Support for "template inheritance" (partials with parameters) `{{<foo}} {{$param1}}...{{/param1}} {{$param2}}...{{/param2}} {{/foo}` following the widely-implemented semi-official specification mustache/spec#75 (@gasche, 58) * Partials are now supported in the `mustache` command-line tool (@gasche, rgrinberg/ocaml-mustache#57) They are interpreted as template inclusion: "{{>foo/bar}}" will include "foo/bar.mustache", relative to the current working directory. * Improve error messages (@gasche, rgrinberg/ocaml-mustache#47, rgrinberg/ocaml-mustache#51, rgrinberg/ocaml-mustache#56) Note: the exceptions raised by Mustache have changed, this breaks compatibility for users that would catch and deconstruct existing exceptions. * Add `render_buf` to render templates directly to buffers (@gasche, rgrinberg/ocaml-mustache#48) * When a lookup fails in the current context, lookup in parents contexts. This should fix errors when using "{{#foo}}" for a scalar variable 'foo' to check that the variable exists. (@gasche, rgrinberg/ocaml-mustache#49)

Add template inheritance spec

ab22750

gasche added a commit to gasche/ocaml-mustache that referenced this pull request Dec 31, 2020

add inheritance specs to the specification testsuite

d423fcb

The specs come from the semi-official specification mustache/spec#75

gasche mentioned this pull request Dec 31, 2020

Hogan-style template inheritance rgrinberg/ocaml-mustache#58

Merged

gasche added a commit to gasche/ocaml-mustache that referenced this pull request Jan 3, 2021

add inheritance specs to the specification testsuite

27a89aa

The specs come from the semi-official specification mustache/spec#75

gasche added a commit to gasche/ocaml-mustache that referenced this pull request Jan 5, 2021

add inheritance specs to the specification testsuite

0d905bc

The specs come from the semi-official specification mustache/spec#75

gasche mentioned this pull request Jan 9, 2021

Could this repository be maintained again? #115

Closed

brabanod mentioned this pull request Jan 17, 2021

Template inheritance not working bobthecow/mustache.php#367

Closed

jgonggrijp requested changes Mar 9, 2021

View reviewed changes

adam-fowler mentioned this pull request Mar 22, 2021

Template inheritance hummingbird-project/swift-mustache#9

Merged

This comment has been minimized.

Sign in to view

chrisclayson mentioned this pull request Apr 14, 2021

OXSvsICt: Use a common layout for all page templates alphagov/di-auth-oidc-provider#14

Merged

jgonggrijp mentioned this pull request Apr 19, 2021

Optional inheritance spec #125

Merged

Danappelxx closed this Apr 21, 2021

jgonggrijp mentioned this pull request Jul 9, 2021

Proposal on how to specify indentation of blocks #130

Closed

psafont mentioned this pull request Nov 24, 2023

[new release] mustache (3.2.0) ocaml/opam-repository#24839

Merged

		- name: Two overridden partials
		desc: Two overridden partials with different content

	desc: Inherit indentation when overriding a partial
	desc: Inherit indentation when overriding a parent

		- name: Override partial with newlines
		desc: Override partial with newlines

Add template inheritance spec #75

Add template inheritance spec #75

Conversation

jazzdan commented Apr 14, 2014

bobthecow commented Apr 14, 2014

groue commented Apr 15, 2014

groue commented Apr 15, 2014

bobthecow commented Apr 15, 2014

bobthecow commented Apr 15, 2014

bobthecow commented Apr 15, 2014

bobthecow commented Apr 15, 2014

groue commented Apr 15, 2014

groue commented Apr 15, 2014

bobthecow commented Apr 15, 2014

groue commented Apr 15, 2014

bobthecow commented Apr 15, 2014

groue commented Apr 15, 2014

bobthecow commented Apr 15, 2014

bobthecow commented Apr 15, 2014

groue commented Apr 15, 2014

groue commented Apr 15, 2014

bobthecow commented Apr 15, 2014

groue commented Apr 15, 2014

bobthecow commented Apr 15, 2014

groue commented Apr 15, 2014

bobthecow commented Apr 15, 2014

groue commented Apr 15, 2014

bobthecow commented Apr 15, 2014

groue commented Apr 15, 2014

bobthecow commented Apr 15, 2014

groue commented Apr 15, 2014

groue commented Apr 15, 2014

groue commented Apr 15, 2014

jgonggrijp left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gasche commented Mar 9, 2021 • edited Loading

jgonggrijp commented Mar 10, 2021

jgonggrijp commented Mar 15, 2021 • edited Loading

jgonggrijp commented Mar 15, 2021

gasche commented Mar 19, 2021

gasche commented Mar 19, 2021 • edited Loading

jgonggrijp commented Mar 19, 2021

gasche commented Mar 21, 2021

This comment has been minimized.

Danappelxx commented Apr 21, 2021

jgonggrijp left a comment •

edited

Loading

gasche commented Mar 9, 2021 •

edited

Loading

jgonggrijp commented Mar 15, 2021 •

edited

Loading

gasche commented Mar 19, 2021 •

edited

Loading