Proposal: Template inheritance

[spullara]

At Twitter we run into a very common case where we want to have one template where we maintain the "bones" of the page and each real page on the site merely replaces sections of that page with appropriate content. We would like to avoid putting any business logic in the bones of the page for including the right content, nor do we want to write custom backing code for each of those pages.

I give up trying to put my code in the issue, here is a gist: https://gist.github.com/1854699

[davidsantiago]

I'm pretty uninterested in the logic-less template stuff, but this strikes me as something that could be really useful to have, and so I'd like to see something like this.

But what strikes me about it is that sub.mustache is not really a mustache template but more of a list of substitutions (which are mustache templates). Is there any way an inheriting template isn't just a list of substitutions? Because if not, why couldn't it just be defined that if an implementation wants to create a mechanism for creating derived templates, it can do that by letting you define (however you like; json? yaml? list of mustache sections?) a mapping of names to mustache-template strings or mustache files and replacing regular interpolation tags with the indicated mustache code when the derived template is loaded? It seems to me (albeit after only like 4 minutes thought) that this would potentially be even more convenient while simultaneously not requiring major changes to the parsers of existing mustaches. And also, it would prevent us from using up two precious special characters to implement this proposal. But, I'm probably missing something in that logic.

[groue]

Two cents:

As @davidsantiago, I regret that sub.mustache is "not really a mustache template", and, more specifically, that we don't know what happens when sub.mustache contains content outside of the substitutions blocks. That alone makes the gist weird, even though the need is pretty clear.

I know about a solution for this problem: Ruby on Rails' one : one template is rendered in one layout.

This gist show hows a Rails-like two-pass rendering could look in Mustache (the syntax mimics Rail's one) : https://gist.github.com/1855111

The main differences with @spullara's gist:

• sub.mustache has no reference to its layout - the rendering code should take care of rendering sub.mustache in the layout.mustache
• the content of sub.mustache is no longer a specific substitution, instead there is a dedicated place for it in the layout.
• there is no default value for any substitution

[spullara]

Thanks for the feedback. sub.mustache is actually a mustache template, the example usage failed to show that in all its glory. Another example that has come up for us is for having similar widgets that are displayed differently in different places but use the same base template. I do agree though that within the {{<super}}, the top level has to be replacements. However, nothing stops you from using the {{<}} syntax anywhere within any template. {{<}} means "replace any blocks in this partial with blocks that i have specified within this tag, then include the partial".

The RoR solution lacks the ability to have multiple layouts in a single page, also I need some external mechanism to connect the two templates. It also doesn't seem to support a large hierarchy of template "classes". For example, we might have:

General Twitter Page < Settings style page < Profile settings page

Re: characters. This scheme can be implemented without introducing new characters through some elaborate lambda gymnastics but it wouldn't be very clear what was going on when you looked at the template. Could be ameliorated by using a naming convention e.g. {{#<super}} and {{#$content}}. My personal opinion is that this is such a widespread issue with real life mustache usages that it just might be worth it.

[groue]

{{<}} means "replace any blocks in this partial with blocks that i have specified within this tag, then include the partial".

Thanks for this sentence. It's quite clear, now.

It's also clear that {{<}} can be seen as a kind of inheritance, and as well as a function call with arguments. More people should know Handlebars.js. In Handlebars, no need for new punctuation, no need for an extension to the spec: I quite think you could just write a simple 'content_for' helper, and write : https://gist.github.com/1855986.

Really, Yehuda Katz's announcement is refreshing: http://yehudakatz.com/2010/09/09/announcing-handlebars-js/
Handlebars.js website: http://handlebarsjs.com/

[spullara]

That is pretty close. The only gap is that when you use multi-level inheritance the higher one in the stack wins instead of the lowest one in the stack. Default content also doesn't work with this model.

[davidsantiago]

OK, yeah, that functionality wasn't clear to me from the initial example. That does leave the "templateness" of sub.mustache intact. However, as you say, there is still the way that the inside of the ">" tag is basically putting forth a map notation for mustache. Mustache has always seemingly been happy to leave maps to the language implementing it, so I still feel a little uncomfortable with this and can't say why. I just wish that the inside of that tag didn't require so much special meaning and new parsing. That is why I still really like the idea of letting this just be a partial with template substitutions for interpolation tags. The spec is already vague about where exactly the implementation should go looking for partials, so this could be very easily slipped into current implementations with a minimum of new syntax, just an expansion of what partials mean, in that partials might have associated files that create substitutions when they are loaded.

Regarding the other character, I was saying that I think you could leave off the $, and just jam the template into the ordinary interpolation tags. This is already kind of what lambdas do, in that a lambda passed in for an interpolation tag will return mustache code that is then parsed and rendered in that context. So there is already a precedent for a regular interpolation tag being used as a site for more template code to be inserted. The difference here would be that instead of this happening due to a function passed into the context map, it's happening because a derived template is instantiating itself (and thus wouldn't need to happen every time the template is called).

[spullara]

The biggest problem with using # is that the identity transformation is so goofy. Even though it looks like you are replacing it with the same thing, they would be interpreted much differently:

https://gist.github.com/1881261

[davidsantiago]

I'm not at all following what you're saying... What is it about the # tags? What is the identity transformation?

[spullara]

Ah sorry, by "identity transformation" I mean a replacement that leaves you with the original. If we reuse # tags we have the issue that to replace the section with something that means the same thing you have to double them.

[davidsantiago]

OK, I see now what you're saying. If we grant the syntax you want for using sections as keys and their values in a partial, the repetition in that example seems clear enough to me. Certainly the identity transformation is not a motivating use case for this feature, and having the source code for it, when it is necessary, shake out in this way doesn't strike me as sufficient motivation for the addition of a new tag type. There are no current semantics defined for sections when they are being used as the sites of template inheritance replacements, so there is some leeway here to posit new rules if that's necessary.

[spullara]

So currently the semantics for # sections is such that if they are not present in the backing code, they are "false". It might make more sense to use {{^}} instead in this case to get something that is closer to the {{$}} behavior. Not sure if I am explaining this well enough, but {{#}} has very different semantics.

[davidsantiago]

I don't know if you're explaining it well enough, but I'm not quite following you. I assume you're talking about the resolution rules hereabouts: https://github.com/mustache/spec/blob/master/specs/sections.yml#L11 Those rules name a few different places to look for the contents of a section tag... is there some reason another place to look couldn't be added to those rules? I'm not seeing it.

[spullara]

So lets take the example where you want a page to evaluate two different ways, once with the default text and once with your modified text. In order for that to work, based on Rule 2), you have to have a singular valued hash matching the name of the section in the top level template so the section renders at all:

2. Walk the context stack from top to bottom, finding the first context
   that is a) a hash containing the name as a key OR b) an object responding
   to a method with the given name.

If you don't have that, the section won't be evaluated. When I extend the top level template, suddenly that "section" is satisfied and it doesn't even reference the section code that I use in the super template. Having to offer specific backing code in order for the template to render properly is what I am trying to avoid by suggesting a declarative template inheritance feature. This is why the sub template isn't just a hash of values but literally replaces the template code in the super template. The real issue is the default for {{#}} is false, not true -- which is why I suggested that {{^}} would be more appropriate.

[davidsantiago]

OK, gotcha, thanks. I had not been considering the use case of wanting to have default code for when nothing is provided by a derived template and there is no desire to have a flag in the context to control whether that default code renders or not. I agree that that's not desirable.

If a partial has {{something}}, {{#something}}{{/something}} or {{^something}}{{/something}}, I thought we'd like to have a way to replace that tag with an arbitrary expansion, but Mustache doesn't currently have a way to name groups of code other than sections and inverted sections. So then you'd have to say: when you're replacing, match the tag names regardless of the tag type. That way you could get a section/inverted section named "something" inserted in the place of {{something}} or whatever. Here we're using sections/inverted sections just for their code-grouping semantic to define the key/value syntax for the replacement part of the {{<}} tag. Although you're right that {{^}} would be more natural for the replacement sites, it's not clear to me which is more natural and meaningful in the key/value replacement syntax, especially when you add on the question of whether they should match the replacement site or be the opposite for the replacement to avoid confusion about "double negatives."

That makes me unhappy, and you already had a solution to that, which was to define a new tag type that allows you to name a group of code, but without the implied "here or not" choice between a section or inverted section. So {{#}} means "Render this group of code if we are given a truthy value for its name," {{^}} means "Render this group of code if we aren't given a truthy value for its name," and {{$}} means "Render this group of code if we aren't given a substitution from a partial, and if we are, render that replacement." This expansion of syntax allows more information to remain "in mustache," as mustache is then expanded to mean more things. This trades away the ability to "replace anything" in the parent template for one that has to be prepared with replacement sites and requires a special idea of what is "default" in the template that is above and beyond just the template itself as it would already exist in mustache as it is today.

I also had a solution to that, which was to not get into the key/value syntax business in the first place and leave those to the implementation's maps. In that world, {{}}, {{#}}, and {{^}} can all be replaced equally naturally when the super (actually just a partial, now) is loaded by the implementation with the use of a map of the tag name->source code replacements that the implementation knows how to find. All that's needed is some rules for how those replacements should be made from the map, assuming this map already exists. This offers "replace anything" capability and retains implementation simplicity by avoiding the addition of any new syntax, but it comes at the cost of requiring that derived templates have some external storage format outside of mustache to store their replacement maps (the spec itself shows that mustache can be very naturally embedded in YAML, for one example). I think we assign different weights to how much we value these things; I don't value having a mustache version of a key/value map that much.

Anyhow, I think your idea for derived templates is great, and I think I'm gonna try adding my version to my mustache implementation just to check it out. I just hope the mustache language stays small and provides a small number of very flexible primitives. The language is pretty much defined by the 5 or 6 special characters that start tags, and I hope the language doesn't expand to many more tags than that, or it'll stop being easy to learn and keep in your head.

[sayrer]

I added this feature to Hogan as well as the Ruby implementation (not submitted yet).

It's useful and helps avoid triple stache usage.

[MattCheely]

In general, I like the fact that the mustache spec is very light, and doesn't add many features, but this is one I'd like to see. In addition to being generally useful, it resolves an issue that has been nagging at me about mustache which is a tendency for markup to get mixed in with logic & data (sort of the opposite of the issue with most templating toos). In particular, this seems to be a problem to me when creating generic re-usable templates that aren't bound to specific data types. Things like form inputs (and the associated markup). Take the following examples: https://gist.github.com/2176799

The thing that bothers me about these templates are that the id, class & href attributes that have to be defined in code to make them re-useable. I would like be able to edit markup unrelated to application data in the templates. With @spullara's inheritance proposal, data could come from the application, and markup could stay in the templates.

[klj613]

inheritance +1

[trans]

-1

To sum up my assessment: "intolerably inelegant".

[spullara]

Would love to hear more trans. It really works very well for things that mustache is very bad at doing without introducing logic to the templates. What in particular do you find inelegant?

[trans]

I give Hogan credit for at least using blocks to designate the template, not just a top-line like dust.js. That makes it a little better than I originally thought. (Documentation on this is bit hard to come by. I had to read some terse tests.)

But I also think:

1. The use of < is a bit too easy to mistake for >.
2. Dummy text clutters up the layout template, the use of blocks (the $ block) are meaningless.
3. The use of $ in the layout template seems rather unnecessary anyway. Without it layout templates would be perfectly reusable with or without template inheritance.
4. Using $ in both the layout template and the content template makes it unclear which is which at a glance.
5. Why $? B/c it's next to # on the keyboard? Try =, which makes the intent much clearer.

Taken alone each of these points might seem rather trivial, but taken all together it leaves me feel'n a bit too perlismy.

[spullara]

On Thu, Jun 28, 2012 at 11:25 PM, 7rans
reply@reply.github.com
wrote:

I give Hogan credit for at least using blocks to designate the template, not just a top-line like dust.js. That makes it a little better than I originally thought. (Documentation on this is bit hard to come by. I had to read some terse tests.)

But I also think:

1. The use of < is a bit too easy to mistake for >.

Suggestions? I think the symmetry between the two makes a lot of sense.

1. Dummy text clutters up the layout template, the use of blocks (the $ block) are meaningless.

That isn't dummy text, it sits in as the default if it isn't overridden.

1. The use of $ in the layout template seems rather unnecessary anyway. Without it layout templates would be perfectly reusable with or without template inheritance.

They are usable without template inheritance. If not overridden, it is
as if they aren't there.

1. Using $ in both the layout template and the content template makes it unclear which is which at a glance.

Believe it or not, it started out as $ to indicate that something
could be replaced and = to set it. This was deemed extraneous by all
who reviewed it.

1. Why $? B/c it's next to # on the keyboard? Try =, which makes the intent much clearer.

$ is the symbol for a variable in many languages. I'm not tied to it.
I'm more than happy to hear alternative syntaxes for the same
behavior.

Sam

Taken alone each of these points might seem rather trivial, but taken all together it leaves me feel'n a bit too perlismy.

Thankfully none of them cut at the meat of the feature but instead are
about syntax.

Sam

---

Reply to this email directly or view it on GitHub:
#38 (comment)

[trans]

Suggestions? I think the symmetry between the two makes a lot of sense.

Yes, I think the symmetry, on its face, does seem like a good idea. But on reconsideration I had to agree with another poster in #44. Using a more distinct symbols would give better "contrast", making it a bit easier to read. My idea was to use a combination of # and >, e.g. {{#> layout}}, but that has it's short-comings too. Really this is the least troubling of my points. I can live with it. (I'd just rather not have to!)

That isn't dummy text, it sits in as the default if it isn't overridden.

We don't have such defaults for normal variable substitutions. So why have them for these? Use inverted sections instead (e.g. {{^$title}}). On the other hand, I am not against these per-se, if they can be optional somehow, b/c most of time the default is a YAGNI. What if it used plain variable substitution (e.g. {{title}}), and only use $ notation if a default is needed. This pattern could then apply to normal variables as well, which would be a more useful feature. I don't think there needs to be a distinction between variable substitution and template substitution, and templates would be more flexible/reusable that way.

They are usable without template inheritance. If not overridden, it is as if they aren't there.

But the default would show up, right?

Believe it or not, it started out as $ to indicate that something
could be replaced and = to set it. This was deemed extraneous by all
who reviewed it.

Really? I'm a bit surprised. I invite the distinction so it is immediately recognizable which is which. Also, what happens if you use template inheritance inside another template inheritance?

Aside, was wondering also if a template block could forgo any "slot definitions", in which case it would default to a special name. e.g.

  <p>{{{yield}}}</p>

  {{< layout}}
    Just content here, no slot defs.
  {{/ layout}}

[klj613]

template inheritance is a good thing and having default blocks help you not to repeat yourself and have it so you can override them. e.g. a sidebar might he overrided only on a few templates.

Twig template system is very nice. It for PHP framework symfony2.

[trans]

I am not disputing there value. I just want to see them be as developer friendly as possible.

With regards to defaults, when you need them then yes it's a good thing. Most of time we don't need them, and we end up with silly filler, like {{$title}}Page Title{{/title}}.

[gw0]

1. The use of < is a bit too easy to mistake for >.

Suggestions? I think the symmetry between the two makes a lot of sense.

What about {{<<super}}? This way it is distinguishable from > and it can also be understood that there is some special kind of input from super (in Bash << represents reading from input until a special line is read).

[gw0]

I would also like to note that template inheritance is the most powerful and useful feature of Django and Jinja2 template engine (check out its goodies). Without template inheritance you mostly end up with chaotic and unreusable templates.

[pvande]

@spullara I'd be interested to know whether your needs would be met by an inheritance pragma...

{{! layout.mustache }}
<html>
  <head><title>{{ title }}</title></head>
  <body>
    {{{yield}}}
  </body>
</html>

{{! content.mustache }}
{{% INHERIT layout yield %}}
<div>Hello, {{name}}!</div>

data = { :title => 'Greetings', :name => 'world' }
Mustache.render File.read('content.mustache'), data

Yielding:

<html>
  <head><title>Greetings</title></head>
  <body>
    <div>Hello, world!</div>
  </body>
</html>

With the INHERIT pragma taking both a template name (resolved like partials) and an (optional) content name. Suspected behavior is as follows:

• Content template is rendered normally.
• Pragma causes the parent template to be fetched and rendered.
  • Parent template is rendered against the the same data content was rendered against
  • Parent template's context stack also includes a reference to the content template, named with the pragma's content name.
• Multiple inheritance is not supported.
• Returned result is the top-level template.

[ghost]

@spullara Did you use helpers with Handlebars?

[spullara]

Sorry, naming confusion. Mustache.java has a small server that renders templates backed by static json files named handlebar. Should probably change the name now.

On Nov 21, 2015, at 7:59 AM, Jamie Hill notifications@github.com wrote:

@spullara Did you use helpers with Handlebars?

—
Reply to this email directly or view it on GitHub.

[ghost]

@groue Based on your description, I've tried to write it as pseudo code to help my understanding... finding it really hard to implement in Tache. Would you mind telling me if I'm on the right track. My main problem is that it's hard for me to keep a global queue as my implementation makes use of template objects for each partial etc; as you will see here.

Render
  If <
    Enqueue
    Render partial
    Dequeue
  If $
    Iterate reversed partial queue
      If not been used yet
        resolver(content of partial override)
      If used
        Flag partial as used

resolver(content):
  For each element in content
    If $
      If name matches
        Flag this as overriden
        return block
    If <
      resolver(parent partial content) or resolver(inner content)
    If >
      resolver(partial content)

[groue]

Hello @thelucid.

My main problem is that it's hard for me to keep a global queue as my implementation makes use of template objects for each partial etc.

I don't use a global queue, and I use objects for partials as well, so this should not be a problem.

When I compare the pseudo code below to https://github.com/groue/GRMustache.swift/blob/Swift2/Mustache/Rendering/RenderingEngine.swift#L94-L99 I think it's OK. You use a queue, when I derive a temporary extended context, but I think it should eventually be the same, and that's why I used the word "queue" in my initial explanations. I hope I was not wrong.

Render
  If <
    Enqueue
    Render partial
    Dequeue

However the code below does not exactly look like the same algorithm as https://github.com/groue/GRMustache.swift/blob/Swift2/Mustache/Rendering/RenderingEngine.swift#L87-L92, because I don't see that you resolve the $block before rendering it.

  If $
    Iterate reversed partial queue
      If not been used yet
        resolver(content of partial override)
      If used
        Flag partial as used

Your resolver function looks pretty similar to https://github.com/groue/GRMustache.swift/blob/Swift2/Mustache/Rendering/RenderingEngine.swift#L251-L319, so it looks OK.

Maybe that the issue you are facing is that those functions should be pure functions, and not modify any shared state. Especially, the flags "block is overridden" and "partial is used" should remain local to each function.

This algorithm is very difficult to track mentally, I agree.

Maybe you can try to make it run in your head with a very simple case, and make your ruby code very verbose so that you can check that it performs as expected.

The Swift code contains tests for the complex cases (why must we test that a partial has not been used yet, for example: https://github.com/groue/GRMustache.swift/blob/Swift2/Mustache/Rendering/RenderingEngine.swift#L221-L233).

[ghost]

@groue I have a working Ruby implementation now (not pushed to Github yet). It passes all your tests except for those where partials and layout partials are place directly within blocks — this causes complications as mentioned by @bobthecow in #75. I'm not sure the additional complexity and additional processing is worth it of the negligible benefit that you get from allowing '>' and '<' tags within '<' content.

@bobthecow where did you get to on allowing this?

[bobthecow]

Mustache.php does not allow partials as direct children of parent tags, for the reasons outlined in #75.

[groue]

What kind of input can't your handle?

[ghost]

@bobthecow Out of interest, have you had any requests for that functionality or needed it in any of your projects to date? Just trying to determine if the additional processing and complexity is worth it, as I currently I have a very simple, clean and fast implementation.

@groue I'm not sure if you mean input on the topic, or input with regards the mustache template?

[groue]

Sorry @thelucid, it's OK, I had a better re-reading of [#75](#75.

OK, so... welcome to the club of the Mustache engines with template inheritance!

[bobthecow]

@thelucid No, I haven't needed nor had any requests for that.

[ghost]

Ok, so finally there is a Ruby Mustache implementation with support for layouts! Tache is actually quite a bit faster than the original implementation and just uses an AST vs. evaling generated code.

These are the tests it passes: https://github.com/thelucid/tache/blob/master/test/fixtures/tests/layouts.json (adapted from GRMustache, thanks @groue).

[groue]

Well done, and thank you, @thelucid ! :-)

[nkurothec]

I have an issue. I am doing the inheritance
{{<../email_base}}
{{$email_title}}Your Title here{{/email_title}}
{{$email_content}}
{{/email_content}}
{{/../email_base}}

The only thing is I have two separate folders and my email_base is outside that folder. When I run my project on Intellij I can see everything properly. But mvn fails
mvn test -DskipTests=false passes
mvn verify -DskipTests=false fails
giving me an error
com.github.mustachejava.MustacheNotFoundException: Template developer/../email_developer_base.html.mustache not found
P.S. Very new to mustache

[spullara]

Sounds like this is specific to mustache.java. I would post to the google
group:

https://groups.google.com/forum/#!forum/mustachejava

On Tue, Dec 22, 2015 at 5:31 PM, nkurothec notifications@github.com wrote:

I have an issue. I am doing the inheritance
{{<../email_base}}
{{$email_title}}Your Title here{{/email_title}}
{{$email_content}}
{{/email_content}}
{{/../email_base}}

The only thing is I have two separate folders and my email_base is outside
that folder. When I run my project on Intellij I can see everything
properly. But mvn fails
mvn test -DskipTests=false passes
mvn verify -DskipTests=false fails
giving me an error
com.github.mustachejava.MustacheNotFoundException: Template
developer/../email_developer_base.html.mustache not found
P.S. Very new to mustache

—
Reply to this email directly or view it on GitHub
#38 (comment).

[ghost]

Guys, how do you get around situations like this? In this case I want to omit an the 'id' block but have no way of doing it:

{{! partial.mustache }}
<section id="{{$id}}{{/id}}" class="content__block">
{{$content}}
  <p>Default content.</p>
{{/content}}
</section>

{{! index.mustache }}
{{<partial}}
{{$content}}
The content.
{{/content}}
{{/partial}}

I'm wondering if we should have a way to conditionally render content, only when a block is supplied (I'm not sure if the '?' is a good idea but not sure what other syntax would make sense), for example:

{{! partial.mustache }}
<section{{$id?}} id="{{$id}}{{/id}}"{{/id?}} class="content__block">
  ...
</section>

Any ideas?

[dasilvacontin]

@thelucid {{#renderId}}id="{{id}}"{{/renderId}}. If it depends on whether the id value is provided or not, {{#id}}id="{{.}}"{{/id}}.

[groue]

@thelucid: why don't you just add a regular {{#conditional}} section around the block that you want to optionally render?

[ghost]

In this case I'm not specifying any data i.e. a front end guy who is only in charge of templates wants to use that layout partial and not specify an id... it should then somehow omit the id html attribute. There is currently no way to do this.

[ghost]

Seems like a common use case.

[spullara]

just include the id= in the the content.

<section {{$id}}{{/id}} class="content__block">

I kind of think of your example is an abuse of the idea of "layout".

[ghost]

@spullara I'm doing that as a workaround but it's not very DRY.

I think it's the perfect use case for layout patials i.e. when you have a chunk of markup that you want to reuse etc. Otherwise I'd just have to repeat all that markup every time meaning that if there were significant changes, they would need to be done with a project wide find/replace.

[bekce]

Hello, can we get a working TL;DR? Thanks

[spullara]

@thelucid I am revisiting this thread as I got a request from someone to conditionally render based on whether a section was overridden. After writing some horrid code to implement it with pragmas I'm think that your {{?id}} syntax might be great for this. Did you ultimately end up using that?

[jgonggrijp]

Implemented in #75 and #125.