Dynamic Partials and Partial Collections through Coercion #54

Open
thelucid opened this Issue Aug 6, 2012 · 124 comments

6 participants

@thelucid

I've just created a pull request over at janl/mustache.js#242

I am using this solution in a couple of projects and it addresses a number of concerns I often hear with regards dynamic partials in Mustache, please see ticket (janl/mustache.js#242) and documentation here:

Really, I urge everyone to have a play with it as it has cleaned up my templates no end.

Kind regards,

Jamie

@devinrhode2

The proposed syntax:

base.mustache
{{#items}}
<p>{{>.}}</p>
{{^items}}

and

base.mustache
{{@items}}

Assume there is a key named 'partial' and then looks up that partial. The idea of dynamic partials is great, but I don't know about assuming a certain key to be present is the most developer friendly approach and the easiest to understand quickly. It could instead by {{>.partialName}} which, when the lookup for the literal key '.partialName' fails, it resolves .partialName to 'text' and looks that up.

It's clearly valuable, but I don't think it fits in the spec for 2.0

@thelucid

See my response here: janl/mustache.js#242 (comment)

As for it not fitting the 2.0 spec, there is currently no way to render partials dynamically which I see as a significant drawback. This is fine for smaller projects as you just add a boolean to each object in a collection such as is_text or is_image and have conditions in the template:

View

{
  items: [
    { type: 'image', url: 'Some URL', is_image: true },
    { type: 'text', content: 'Some text', is_text: true }
  ]
}
{{#items}}
  {{#is_text}}
    <p>{{content}}</p>
  {{/is_text}}
  {{#is_image}}
    <p><img src="{{url}}"/></p>
  {{/is_image?}}
{{/items}}

But... as soon as this goes above a couple of types, things get hard to manage. The whole premise of Mustache is to keep logic out of the template and this reads to me as a bunch of if statements.

Wouldn't it be nicer to be able to do:

base.mustache
{{@items}}

text.mustache
<p>{{content}}</p>

image.mustache
<p><img src="{{url}}"/></p>

So much cleaner, don't you think?

Sorry to labour my point, but I'm using this in production and it's changed my life :)

@groue

@thelucid As usual, when one's code contains too many if, it means that you need a to inject a little object-oriented design. Let's replace your is_text, is_image "queriable" attributes by some "performing" attributes, and use Mustache lambdas:

{
  items: [
    { url:     'Some URL',  html: function() { return '<p><img src="{{url}}"/></p>'; } },
    { content: 'Some text', html: function() { return '<p>{{content}}</p>'; } }
  ]
}

base.mustache:
{{#items}}
  <p>{{{html}}}<p>
{{/items}}

And, if you absolutely want to use partials:

{
  items: [
    { url: 'Some URL',      html: function() { return '{{>image}}'; } },
    { content: 'Some text', html: function() { return '{{>text}}'; } },
  ]
}

base.mustache
{{#items}}
  {{{html}}}
{{/items}}

text.mustache
<p>{{content}}</p>

image.mustache
<p><img src="{{url}}"/></p>

And voilà, no need for a spec update!

@thelucid

@groue Ew, I'm sorry but this is exactly why the functionality is needed. Returning chunks of html from the view is in no way a good solution to the problem... I would hope I'm not alone on this.

This is in fact a perfect illustration as to why we need a clean solution to this problem, if hacks like @groue is suggesting are being suggested then something is fundamentally wrong.

@groue

@thelucid You call it a "hack" because you see data as a model model object, not as a view model. I can understand you: Mustache has always been ambiguous on that subject: advocating simplicity, saying "just give me your model objects, I'll render them", while defending itself against missing features with arguments like "just add the missing keys to your view model, you shouldn't have provided a model in the first place anyway".

Mustache spec issues history is full on this kind of situations, if you read them carefully.

So, in a way, I gave an answer that is typical of the "Mustache 1" spirit. Be free to suggest improvements for Mustache 2. But dont' call "hacks" ways to use Mustache that you just don't happen to like.

@thelucid

@groue I'm sorry, I just don't see that returning a chunk of html from a view object (be it a view model or a data model) is a maintainable solution. Where do you draw the line, you may as well bypass Mustache completely and just interpolate strings.

I would much rather do:

base.mustache
{{@items}}

text.mustache
<p>{{content}}</p>

image.mustache
<p><img src="{{url}}"/></p>
@groue

@thelucid Have you read my message? Check after the "And, if you absolutely want to use partials:" sentence.

@thelucid

@groue Yes, I have read the message and there are a few problems with the suggested partial solution:

  1. It's ugly but that's just my opinion.
  2. It's not DRY, you have two near-identical functions.
  3. It's unmaintainable, i.e you will have to add a function for every new partial.
  4. It's open to injection attacks since you are using the {{{html}}} approach.

Injection example:

{
  items: [
    { url: 'Some URL',      html: function() { return '{{>image}}'; } },
    { content: 'Some text', html: function() { return '{{>text}}'; } },
    { html: '<script>alert("I am hijacking your page")</script>' }
  ]
}
@groue

It's ugly but that's just my opinion.

As much as the is_text and type attributes, and more OO.

It's not DRY

Single interface + Liskov substitution principle is no DRY for you? Gosh, I can't see how more DRY it could be.

It's unmaintainable, i.e you will have to add a function for every new partial.

Your solution must keep synced the type value and the partial name, which is just the same chore.

It's open to injection attacks since you are using the {{{html}}} approach.

Is it? open a new issue, and ask for a removal of the triple mustache.

Listen: I won't defend my proposal more than necessary. Actually, I'm quite sure you didn't know about the lambda before I introduced them to you. Think more about the tool, make sure you know every tiny details: Mustache is more rich that it looks. Maybe you'll find a solution that conforms to your standards with the spec as it is.

@thelucid

I'm at a loss as to why you're against a clean solution to dynamic partials.

As much as the is_text and type attributes, and more OO.

That is exactly why I am suggesting such a feature, it is dry and concise.

Single interface + Liskov substitution principle is no DRY for you? Gosh, I can't see how more DRY it could be.

You are still duplicating a function which is not DRY.

Your solution must keep synced the type value and the partial name, which is just the same chore.

Yes, but you are just specifying the partial to render, not a function that actually renders a chunk of html.

Is it? open a new issue, and ask for a removal of the triple mustache.

The triple mustache doesn't escape the value by design, that's what it's for and how it works. So yes, your example is open to injection attacks... try it if you don't believe me:

View

{
  html: '<script>alert("See, I have control");</script>'
}

Template

{{{html}}}

I did know about lambdas before you mentioned them (that's a little patronising, especially given your lack of understanding when it comes to the triple mustache), I just don't see them as a good fit for this particular problem.

We'll just have to agree to disagree on this one.

Maybe you'll find a solution that conforms to your standards with the spec as it is.

There is currently no way to render dynamic partials other than the suggestions we have already exhausted here, hence this ticket.

@thelucid thelucid referenced this issue in janl/mustache.js Sep 18, 2012
Open

Dynamic partials and partial collections #242

@groue

I'm not against your solution. I just wanted to make sure everybody knows that there are already existing solutions. Usually, people ask for a feature because they don't know their tool enough. Sorry if I have patronized.

About the triple mustache: they allow injection in all situations, not only in the one we are talking about here. What I meant was that the topic of injection should hence not pollute this topic, and that you can ask for the removal of the triple mustache in some other issue.

@devinrhode2

I think @groue has a great point. This is already possible, therefore it's best not to add a feature for it.

I vote no.

@thelucid

It's possible to drive a nail into a block of wood with a mallet but it's not necessarily the best tool for the job.

@devinrhode2

I think letting{{>string}}first lookup the partial named 'string' - and then fallback to looking up the value of the string would certainly be acceptable. This would work perfectly without changing anything, only at a point of failure does it kick in.

And with @bobthecow's suggestion of moar dot notation hotness, you could do{{>.string}}

@thelucid

Yes, that's not too bad, what about the {{@collection}} shorthand?

@thelucid

The only potential problem with that solution is that it's a little ambiguous i.e. the same syntax does two different things. Having a separate syntax makes it easier to see what is going on.

@bobthecow
Mustache member

That should be explicit, if that's what you want. Otherwise you're opening yourself up for exploits. Maybe use a "variable dereferencing" type token? {{> *string }}

@thelucid

What about using the @ symbol for everything related to dynamic partials e.g.

{{#items}}
{{>@partial}}
{{/items}}

or shorthand that assumes a partial key:

{{@items}}

I would personally prefer it being implicit, going with a convention whereby a partial key is always assumed when using dynamic partials i.e.

{{#items}}
{{@}}
{{/items}}

and shorthand:

{{@items}}

The @ symbol then means "render an object or collection of objects using their partial keys". Of course, the key doesn't necessarily have to be partial, perhaps > would be a better fit to denote a partial? ...hmm, maybe not, partial does what it says on the tin.

Has anyone tried a few examples using my patch to mustache.js? The {{@collection}} shorthand really is a joy to use, maybe that's all that's needed and we abandon the longhand.

@devinrhode2

Not a fan of using@ :(

@bobthecow Could you give an example of being exploited?

We could also use another interesting syntax:

{{> {{string}} }}

It seems the only way we dodge ambiguity is by having a special char or marker that means lookup the value of this string not the name 'string'

@thelucid

%? Looks a bit like repeating partials if you tilt your head to the left :) {{%items}}.

In response to the dereferenceing thing, & is commonly used for this in programming languages e.g. {{> &thing}}.

@devinrhode2

{{> {{string}} }}is kinda interesting because it presents the idea that anything can be dynamic lookup. *could be just as universal.

@devinrhode2

Essentially we're talking about the same thing - some symbol to 'dereference' the variable. I really like @bobthecow's* because that's exactly what it means in any non-garbage collected language. (C, C++, etc..)

@thelucid

Not keen on the taches in taches thing, wouldn't that choke the parser?

I'm currently leaning toward:

{{#items}}
{{> &partial}}
{{/items}}

with a shortcut of:

{{%items}}

or

{{@items}}
@devinrhode2

The{{> {{string}} }}syntax would require a more intelligent parser... compiling it would also be a bit more complicated too, I'm not very big on it.

I wonder how a lambda/filter/helper could assist you with a shorthand version...

What does everyone think of having a deference operator, like*?

@thelucid

I'm not against using * although prefer & as it's more Rubyish and reminds me of things.map(&:name) which does exactly what we want.

With regards the shorthand version, in practice this is more useful than the longhand. In fact, I'm using this setup in a couple of projects and have not actually had a need for the longhand version over the shorthand. The resulting templates are super clean and concice.

Let's say you have a page with main content and sub content, and each contains a collection of objects, this becomes (using @ here but could use % instead):

<html>
<body>

<div class="main">
{{@main}}
</div>

<div class="sub">
{{@sub}}
</div>

</body>
</html>

I would be more interested in the shorthand making it in over the longhand, be it {{%items}} or {{@items}}. It's a super simple patch, only requiring a few lines of code in return for powerful dynamic partial functionality.

@groue

@thelucid : are you proposing to reserve a special key partial for your shorthand notation?

@thelucid

Not to necessarily reserve it but the convention would be that when rendering a collection of partials, use the partial key. Have a play with my fork of mustache.js, you'll see how nice it is to use, it just works.

I do quite like the {{> *partial}} or ideally {{> &partial}} notation too, however in practice the shorthand gets far more use.

@groue

@thelucid : Don't play on words: you are indeed "blessing" the partial key it for some special purpose of your own convenience.

Of course, you are feeling it is a bad idea. And you are right: it's a big threat on compatibility. Each time a special key is added to the spec, one has to bump the Mustache version by a major increment (Mustache 2.0, 3.0, etc.) since it could break existng code.

You can give up this idea right away - it will never enter the spec.

@thelucid

I'm not playing on words, this functionality is backwards compatible, if you took the time to think about it or try out the patch before mouthing off you would see that. I'm just throwing some ideas out there that I have found to be hugely beneficial in my own projects and that I thought others could benefit from.

This ticket has thrown up some interesting discussion around the {{> *partial}}/{{> &partial}} syntax so I believe it to have been worthwhile. Far more worthwhile than your suggestion of returning a chunk of HTML.

Your negativity is getting on my nerves and don't tell me what or what not to do. Go find another ticket to troll.

@thelucid

p.s. what makes you think that I think it's a bad idea?

@groue

It is not backward compatible, since a key has become blessed. Code written before the change may use this key. If the user updates the Mustache engine, the rendering will be different.

You will tell me: "this will not happen, since the user has to explicitely opt-in for the shorthand notation. Hence the change is backward compatible, and you are a bad troll".

Sure. Rigorously, you are right. But sooner or later we'll have many smart guys that will bless more punctuation signs and more keys, for their own convenience, and compatiblity will blow up.

Second reason why key blessing is bad idea: user's data is user's data. Don't mess with it, for your own convenience.

@thelucid

What's with all this "key blessing" nonsense. Existing templates will work just fine, you can use the partial key as much as your heart desires and the shorthand just introduces a convention whereby a partial key is used to specify the partial for each item (how often have you actually had a partial key in a view object? ...I would guess never). It's not for my own convenience specifically, it's to allow Mustache users to render a collection of items using specified partials, a pretty common scenario.

Rails uses conventions all the time and that's what makes it such a powerful framework, are you saying that Rails controllers shouldn't assume a show or index method just incase someone wants to use it for something else?

If you are so against convention, how about something like:

View

{
  items: [
    { partial: 'image', url: 'Some URL' },
    { partial: 'text', content: 'Some text' }
  ]
}

Template

{{items >*partial}}

or

{{items >&partial}}

Not too bad, but also not too pretty, that's what happens when you don't have conventions.

This does mean that you could also render a collection with an explicit partial too which is quite interesting:

This:

{{#items}}
{{>my_partial}}
{{/items}}

Could be declared as:

{{items >my_partial}}
@groue

how often have you actually had a partial key in a view object? ...I would guess never

This is typically the behavior I try to avoid for myself : thinking for other people, and believing that my case is the case of everybody. Should a single people, a single user (we are doing user interface here) use a blessed key without knowing it is blessed, his rendering will fail in a confusing and non trivial way, leading to a painful debugging session. This would be bad for that user, and bad for Mustache. Again, for your own convenience - I'll repeat as often as needed that Mustache 1 already provides the feature everybody here pretend is missing.

Your comparison with Rails is interesting, but I have nothing against conventions. I'm a very happy Rails user myself. I would have used the arguments however, should you have been DHH in person. Namely: don't mess with user data because you don't own it, and don't bless keys for backward compatibility's sake.

@groue

A new proposal:

What about changing the regular partial syntax from {{> foo }} to {{> "foo" }}, so that {{> foo }} now means "render the partial whose name is stored at the key foo", just like {{ foo }} means "render the value whose name is stored at the key foo" ?

It would be much more consistant than Mustache 1, and make useless the need for a dereferencing syntax. What do you all you think?

@thelucid

Did you read my alternate suggestion that addresses all your concerns? Not as clean but that's what happens when you don't have conventions.

I'm not trying to think for other people, just suggesting a clean solution to dynamic partials, that's all... not some big conspiracy.

I'll repeat as often as needed that Mustache 1 already provides the feature everybody here pretend is missing.

I'll repeat as often as needed that returning a chunk of HTML is not a proper solution to dynamic partials.

What about changing the regular partial syntax from {{> foo }} to {{> "foo" }}

From someone who's worried about breaking compatability, now this does break comparability.

Does anyone else have a view on the discussion so far as I'm losing the will to live?

@groue

@thelucid : you did not understand my concern about compatibility. Here we are talking about Mustache 2, so we can break stuff if we want. Not ideal, but we can. However, the introduction of a blessed key would provide a hazardous pattern: each enw feature that would "need" a new blessed key would have to bump Mustache major version. Even tiny features that would only deserve a minor bump. That is the only concern with blessed keys. And if there is something in these sentences that you do not agree with, I'll be happy explaining until you understand that blessed keys are plain wrong, however nice they look at first sight. Nothing personal here : keep you will to live.

@devinrhode2

@groue in regards to{{> "foo" }}, if we said that{{> foo }}, upon failure, would fallback to looking up the partial by the name of this key ('foo') instead of searching the it's value as a string this would work. Interestingly, it's the flip of a dereference operator.

I have to agree with groue on the shorthand/special key, it's just not getting into the spec.

@groue

Did you read my alternate suggestion that addresses all your concerns? Not as clean but that's what happens when you don't have conventions.

You are focusing on the rendering of dynamic partials for collections. You are trying here to solve two problems in the same time. What about focusing on dynamic partials first, and then see how it fits with collections?

@thelucid

@groue I understand what you are saying but they are only "blessed" (as you call it) when you explicitly use a feature that makes clear it's intent in the documentation.

What about the {{>static_partial}}, {{>*dynamic_partial}} and {{items >static_partial}} and {{items >*dynamic_partial}} notations? Doesn't this address your concerns?

I personally don't see the need to be quite so explicit but it addresses your concerns.

@groue

in regards to{{> "foo" }}, if we said that{{> foo }}, upon failure, would fallback to looking up the partial by the name of this key ('foo') instead of searching the it's value as a string this would work

@devinrhode2 the problem would be that your feature would be backed on a failure, that is to say a condition that may be unexpected by the programmer. Expect hours of painful debugging, here, again.

On the opposite, {{> "foo" }}, {{> foo }} and {{ foo }} all say a single thing at one time, and are very consistent. One could say that partial tags used to contain a "template name". Now they contain an "expression" just like variable and section tags: {{> partial }}, {{> item.partial }}, {{> first(items).partial }} (using parentheses-based filters).

@groue

Of course, a consequence of {{> "foo" }} vs the dereferencing {{> foo }} would be that {{ "foo" }} would have to render "foo". This is the introduction of literals in Mustache. I expect some people will not like that. Still, this is an elegant solution to the dynamic partial "problem" (admitting there is one in the first place).

@thelucid

@devinrhode2 Fair enough on the shorthand, at least we're in agreement that there needs to be a better way to render dynamic partial than returning a chunk or html.

I'm not keen on one syntax having two possible outcomes i.e. falling back to a key if a partial is not matched. In which case, I vote for:

Dynamic partials:

{{#items}}
{{>*partial}}
{{/items}}

I guess you're not taken by the alternative to {{%items}} or {{@items}}?

{{items >*partial}}

...I was just trying to come up with a way of not always having to write the open and closing tags as it's quite a common pattern.

@groue

What about the {{>static_partial}}, {{>*dynamic_partial}} and {{items >static_partial}} and {{items >*dynamic_partial}} notations? Doesn't this address your concerns?

Yes. However, as an implementor, I would not be happy implementing the shortcut notation. It's there only to fulfill a very specific use case, and I would feel I spend time for a very little user benefit. As soon as you want to render a collection with some decoration, the short syntax becomes useless, leaving everybody using the "long" syntax. Also, the "long" syntax is {{#items}}{{> ...}}{{/items}}... Is it really so long, frankly?

@thelucid, as a Mustache implementor, what I've seen is that users are amazingly various, surprising, and rich. They always have needs I would have never imagined. And it's a joy when you see that all you have to do is to guide the users in your library, so that they can use it in order to accomplish their goal. I'm really against feature bloat. Because it never stops. Some guy will want a shortcut for this. This guy will want a shortcut for that. What they need is a library that can take many shapes, that can host many patterns, that can fit the user's needs, a library that does not need an update when some guy has some crazy idea. That's why I sound so conservative.

@thelucid

@groue Fair enough, I guess I've been in the Rails community for too long where creating shorter, more concise and convenient ways of doing things is ingrained (I created "sexy validations" in ActiveModel core for example).

Over time, the partial syntax in Rails went from:

<%= render :partial => "item", :locals => { :item => @item } %>

to...

<%= render :partial => "item", :item => @item %>

to...

<%= render :partial => @item %>

to...

<%= render @item %>

And collections of partials from:

<%= render :partial => "item", :collection => @items %>

to...

<%= render :partial => @items %>

to...

<%= render @items %>

If we had an inflector in Mustache we could do:

{{>items}}

...and it would automatically look for an item partial but I guess an inflector is going too far and still wouldn't account for dynamic partials.

@groue

@thelucid I'm a happy Rails user, and also an Objective-C developer. Both environments focus on expressivity, with a concise variant on the Rails side, and a verbose variant on the Obj-C side. Now this can explain our "culture clash" :)

If I sum up, we have two subjects:

A: The syntax for static and dynamic partials, with two options so far:

  1. {{> static }} and {{> &dynamic }} or {{> *dynamic }}
  2. {{> "static" }} and {{> dynamic }}

The first introduces a special syntax for "dereferencing". The second has no need for "dereferencing" concept, is more consistent with variable and section, but introduces literals.

B: the questionable need for a collection shortcut, and whether it would be a good idea to "bless" some keys in order to help shortcuts.

My take on the subject is already known. Special keys are a big potential threat on backward compatibility, for a debatable benefit: the "long" version works, for Christ's sake. BTW, what are the special methods that Rails read in order to implement <%= render @item %> and <%= render @items %> ?

@thelucid

@groue Rails uses the class name which we obviously don't have and makes use of an inflector to get the singular which we also don't have.

the "long" version works, for Christ's sake

True, I'm just not a fan of the "If it ain't broke don't fix it" mentality, and instead prefer progress. As I said, you can drive a nail into a block of wood with a mallet but a hammer would probably be your best bet.

My vote on subject "A" is most definitely option 1 as I really don't like the idea of introducing literals.

We're obviously not going to agree on a shorthand ("B") which is a shame as I'm using (and will continue to use a variant of) {{@items}} in a couple of projects as it cleans up templates no end.

I guess the real way forward, after we have agreed on the dynamic partial syntax is to introduce a way for developers to register app specific functionality, so that crazy people like me and those trying out new ideas can do:

Mustache.Renderer.register('@', function(name, context, options) {
  // My custom functionality
  return 'the output';
});

This could also cleanup the internals.

See: janl/mustache.js#242 (comment)

@groue

Rails uses the class name which we obviously don't have and makes use of an inflector to get the singular which we also don't have.

Yes indeed. No magic special blessed key is needed. And that's why it's OK.

The only "blessed" name I can think of right now in Rails is the to_param method. Convenient, but not quite elegant since suddenly models have their word in the URLs. Anyway, it's convenient, let's face it. And in this case, yes, it's quite unlikely that a user would define his own to_param method for some other purpose. And should he, anyway, he would be wrong, since he committed into Rails: by using Rails in the first place, a framework, he has a lot of documentation to read, in order to design classes that fit in the framework. As a side effect, all classes designed for Rails are locked in, now. I don't quite remember this time, but porting from Rails from/to Merb wasn't very easy, I guess.

But Mustache is not a framework. It's a dumb template engine. A mere tool.

My vote on subject "A" is most definitely option 1 as I really don't like the idea of introducing literals.

I usually try to tell you why I don't like some ideas. It would be nice of you to do the same.

Mustache.Renderer.register('@...

It would sure be nice to send out all this painful Mustache to the user. Not an easy task, though, especially when you want to keep intact the hosting language agnosticism of Mustache. Of course, you may want as many Mustaches as users, and ruin the cohesion of today's Mustache landscape.

You know, my GRMustache is quite an old library now, it may be two years old. Its API has continuously evolved, so did its feature set. But no features were added. No. instead I have increased the expressiveness of the library. So that users can extend, with their user-land code, the basic Mustache engine. Want to render 0-based indexes? 1-based indexes? Want to render something every odd elements? Want to format numbers, dates? Need a default values for missing ones? Need to debug your templates? Need to localize portions of your templates? Localization with parameters, so that the template renders "Arthur and Barbara are friends" or "Arthur et Barbara sont amis" ? Check. Done. All of this is based on a genuine Mustache 1 core, and a few carefully designed hooks so that the user injects the code Mustache 1 is missing.

I have quite an experience on this Mustache.Renderer.register('@... topic. I'd be happy discussing it.

@thelucid

The only "blessed" name I can think of right now in Rails is the to_param method.

It's quite common and even reccomended to override the to_param method in Rails for example to return a slug to be used in urls.

I actually like the idea of a to_partial and to_string convention. The to_string would be useful in cases like:

View

{ price: { currency: '£', value: '20.00', to_string: '£20.00' } }

Template

Currency: {{price.currency}} <- '£'
Value:    {{price.value}}    <- '20.00'
Price:    {{price}}          <- '£20.00' (to_string)

or

{{#price}}
Currency: {{currency}} <- '£'
Value:    {{value}}    <- '20.00'
Price:    {{.}}        <- '£20.00' (to_string)
{{/price}}

or partials...

View

{
  people: [
    { first_name: 'John', last_name: 'Lennon', to_partial: 'dead' },
    { first_name: 'Paul', last_name: 'McCartney', to_partial: 'alive' },
  ]
}

Template

{{#people}}
{{>.}}
{{/people}}

or

{{@people}}

The to_partial and to_string keys could even be > and . respectively.

I usually try to tell you why I don't like some ideas. It would be nice of you to do the same.

Liquid templates use literals and the point of literals are to be able to insert data into the template. As the premise Mustache is to be logicless and to keep the data in a view object, the introduction of literals doesn't seem a good fit. Next people will want to perform operations on those literals like in Liquid which could end badly.

I have quite an experience on this Mustache.Renderer.register('@... topic. I'd be happy discussing it.

I'm definitely up for suggestions on how to hook into the language as I think this would be a good move for Mustache and provide a nice way to showcase new functionality as well as providing a way for developers to implement their own project specific functionality.

@groue

to_partial

You know my take on why special keys are bad (see my comments above)

Liquid templates use literals and the point of literals are to be able to insert data into the template. As the premise Mustache is to be logicless and to keep the data in a view object, the introduction of literals doesn't seem a good fit. Next people will want to perform operations on those literals like in Liquid which could end badly.

Go read https://github.com/mustache/spec/wiki/%5BDiscussion%5D-Logic-Free-vs.-Non-Evaled, including my answers, please. Add you own answer to the wiki if you want. And then come back with more explanations of your opinion, if it still stands.

I'm definitely up for suggestions on how to hook into the language [...]

Sure, here some more reading for you! Those suggestions are all written down in GRMustache documentation. Since you like shortcuts, go check the sample code links. Also don't miss the lambda API, which fixes a few flaws of genuine Mustache lambdas, the filters API, and the mother of all hooks, the template delegate API - now GRMustache recommends using filters, but the delegate was the way to go before their introduction.

@groue

All, I've got a fresh new proposal for you.

It's inspired by the fact that the expressiveness of GRMustache comes from its API, not from the Mustache language itself.

So here it is: let's have {{items}} render the concatenation of dynamic partials, by updating the spec with those four sentences:

  1. An implementation MUST render of {{items}} the same way as {{#items}}{{.}}{{/items}} whenever the value attached to items is an enumeration.
  2. When rendering {{value}}, an implementation must fetch the value attached to the key value. If this value is a lambda, or can be automatically converted to a lambda, the rendering MUST be the result of the lambda call.
  3. An implementation SHOULD provide an API for letting the library user specify the automatic conversion to lambda described at point 2.
  4. An implementation SHOULD provide an API for letting the library user specify the automatic conversion to lambda described at point 2 via a partial name.

Now let's see how it fits your needs.

All examples below will use the same templates set:

base.mustache
{{items}}

text.mustache
<p>{{content}}</p>

image.mustache
<p><img src="{{url}}"/></p>

Let's first use a Mustache implementor that only implement MUST rules of the spec:

// User has to provide lambdas in the first place

var text_item = function() { return "{{>text}}"}
text_item.content = 'Some text'

var image_item = function() { return "{{>image}}"}
image_item.url = 'Some URL'

var data = { items: [text_item, image_item] }

Let's now use a Mustache implementation that implements rule 3, but not rule 4. Its documentation says: "If an object has a to_lambda function property, this function is used for automatic lambda conversion (see Mustache doc)."

// User complies with his Mustache implementation documentation by setting
// the to_lambda property:

var data = {
    items: [
        { content: 'Some text', to_lambda: function() { return '{{> text}}' }},
        { url:     'Some URL',  to_lambda: function() { return '{{> image}}' }}
    ]
}

Let's now use a Mustache implementation that implements rule 3 and 4. Its documentation says: "If an object has a to_partial string property, this value is used for automatic lambda conversion (see Mustache doc)."

// User complies with his Mustache implementation documentation by setting
// the to_partial property:

var data = {
    items: [
        { content: 'Some text', to_partial: 'text' },
        { url:     'Some URL',  to_partial: 'image' },
    ]
}

Now, @thelucid, see how this approach is powerful:

A Javascript implementation may state in its documentation: "optional features such as automatic partial and lambda conversions are not supported. You have to provide lambdas objects".

A Javascript implementation may state in its documentation: "in order to get automatic partial conversion, provide the to_partial key".

A Ruby implementation may state in its documentation: "automatic partial conversion is automatic and based on the class of your object"

A Java implementation may state in its documentation: "automatic partial conversion is based on the interface AutomaticMustachePartia which defines the getMustachePartialName method".

An Objective-C implementation may state in its documentation: "automatic partial conversion is based on the conformance to the MustacheMustachePartial protocol, which defines the mustachePartialName method".

See? Now if a library wants to mess with the user data, by defining special keys, it can. This is now its own problem. Some other implementations may be able to provide a much cleaner way to implement the dynamic partial feature.

@thelucid

I definitely like the idea of {{items}} doing the same as {{#items}}{{.}}{{/items}}. So in essence, what you are saying is that if all steps are implemented, we will be able to do:

View

var data = {
  items: [
    { content: 'Some text', to_partial: 'text' },
    { url:     'Some URL',  to_partial: 'image' }
  ]
};

Templates

base.mustache
{{items}}

text.mustache
<p>{{content}}</p>

image.mustache
<p><img src="{{url}}"/></p>

If this is the case, it's awesome, exactly what I was suggesting (minus the @ sign)... we may have found our common ground :)

Presumably, the following would also be possible:

base.mustache
{{#items}}
{{.}}
{{/items}}

text.mustache
<p>{{content}}</p>

image.mustache
<p><img src="{{url}}"/></p>

For completeness, I'd also like to see a step 5 for to_string so that the following is possible:

View

var data = { price: { currency: '£', value: '20.00', to_string: '£20.00' } };

Template

Currency: {{price.currency}} <- '£'
Value:    {{price.value}}    <- '20.00'
Price:    {{price}}          <- '£20.00' (to_string)

or

{{#price}}
Currency: {{currency}} <- '£'
Value:    {{value}}    <- '20.00'
Price:    {{.}}        <- '£20.00' (to_string)
{{/price}}

My only question would be, why not make 3 and 4 MUST's? This is such useful functionality.

I guess all this functionality could come under a "Coercion" title? I am really quite excited by this, it's exactly what I've been pushing for as I've seen the benefits.

From a users perspective, the ability to be able to supply a to_partial or to_string key alone is invaluable.

@thelucid

One slight worry with this... won't the html in the partials get escaped? It's late and not entirely sure so just making a note of it here.

I should probably give my reasoning for saying 3, 4 & 5 (to_string) should be MUST's... the Ruby implementation for example could just have a default to_partial method like so:

def to_partial
  self.class.name.downcase
end

That way it's default implementation will use the class name when determining the partial but gives the user the ability to override it when needed.

Where I have suggested a to_string key, maybe this should be to_value to save confusion with Ruby's to_s (unless of course we just go with to_s, open to suggestions), so we would have to_value and if not available then fall back to to_partial.

@groue

Yes, there are details to set up. But, please, stop mentioning the special keys to_xxx: they are still wrong, they can be avoided in most languages but JS, and the will definitely not enter the spec if we go this way. We should stop mentioning them, and talk about "automatic lambda coercion" or "automatic partial coercion". Please.

@devinrhode2

Don't know why I didn't mention this, but there's another issue thread on dynamic partials. We should all probably read through all that if you guys haven't already, I know I haven't

@thelucid

@groue Apologies, I was referring to the JavaScript implementation hence the Javascript examples. I was only reiterating what you had said with regards to_partial and suggesting a to_string or to_value sibling.

The way I see it, having say a to_partial and to_value (or equivalent language variant) is the most flexible way of handling this. Without them, there will be no way for a user to override the default Ruby implementation behaviour for example.

If you don't want me to reference the to_ keys, we can call it "automatic partial coercion" and "automatic value coercion" but for the JavaScript version at least, this equates to the same functionality. Why not skip the whole to_lambda rigmarole and just have a to_partial and to_value convention?

I would simplify the spec sentences with:

  1. An implementation MUST render of {{items}} the same way as {{#items}}{{.}}{{/items}} whenever the value attached to items is an enumeration.
  2. An implementation MUST provide an API for letting the library user coerce an object to a string.
  3. An implementation MUST provide an API for letting the library user coerce an object to a partial key.

@devinrhode2 Just had a read through #49 (I assume this is the ticket you mean). It is trying to achieve a different goal and is probably better named "Layouts".

@groue

@thelucid I'm so happy we eventually settled something.

About the MUST/SHOULD : Since the feature of rendering a collection of partials can be achieved without the coercion, by just providing a specially crafted data (see the first of three hypothecical implementations at comment #issuecomment-8720055, I don't want to force implementors to implement coercion. Put in another way, a Mustache implementation that does not provide coercion is less easy to use, but still a Mustache implementation, a piece of work that deserve the Mustache label. Hence the SHOULD. Users must be respected, and so do implementors.

About getting rid of the automatic lambda coercion, and keeping only the automatic partial coercion: automatic lambda coercion can be handy. Think of Ruby's to_proc: it's so handy that Symbol#to_proc has eventually entered the language itself, and now we all map(&:foo) like crazy. Plus automatic partial coercion can be based on it. So I'd rather keep both of them. I'll try to imagine some nice uses for automatic lambda coercion, in order to convince you of their usefulness.

@thelucid

@groue Great, we're in agreement. Nothing better than a heated debate to get results :)

How do you feel about "automatic string conversion", to_string in the case of JavaScript? This fits in nicely with "automatic lambda conversion" and "automatic partial conversion" don't you think?

I'm unsure wether to call it "automatic string conversion" or "automatic value conversion" so that it doesn't clash with existing to_s or toString implementations, to_value might be a better choice.

Where to from here? I'm loving the {{items}} being equivalent to {{#items}}{{.}}{{/items}} thing... so, are we agreed on:

  1. An implementation MUST render {{items}} in the same way as {{#items}}{{.}}{{/items}} whenever the value attached to items is an enumeration.
  2. An implementation SHOULD provide an API for letting the library user coerce an object to a string.
  3. An implementation SHOULD provide an API for letting the library user coerce an object to a partial key.
  4. An implementation SHOULD provide an API for letting the library user coerce an object to a lambda.

Examples based on a hypothetical JavaScript implementation:

View

var data = {
  items: [
    { content: 'Some text', to_partial: 'text' },
    { url:     'Some URL',  to_partial: 'image' }
  ]
};

Template

base.mustache (syntax 1)
{{items}}

base.mustache (syntax 2)
{{#items}}
{{.}}
{{/items}}

text.mustache
<p>{{content}}</p>

image.mustache
<p><img src="{{url}}"/></p>

View:

var data = { price: { currency: '£', value: '20.00', to_string: '£20.00' } };

Template

(syntax 1)
Price:    {{price}}          <- '£20.00' (to_string)
Currency: {{price.currency}} <- '£'
Value:    {{price.value}}    <- '20.00'

(syntax 2)
{{#price}}
Price:    {{.}}        <- '£20.00' (to_string)
Currency: {{currency}} <- '£'
Value:    {{value}}    <- '20.00'
{{/price}}

How does everybody feel about this?

@groue

@thelucid The need for "string coercion" comes from nowhere. I mean, it's already there in all Mustache implementations. Besides, should a hackers want to render an object in a funny way, he already has lambda coercion, that is pretty generic. Before adding something, see how your existing tools can help already.

Every good rock climber will tell you that you can't climb unless you trust your shoes: right now you are building a crane.

@thelucid

@groue It doesn't come from nowhere, it's a common pattern in most, if not all languages and gets around the whole filter debate for example (no special pipe syntax needed):

var view = {
  title: {
    to_string: 'Hello World',
    lowercase: 'hello world',
    uppercase: 'HELLO WORLD'
  }
};
{{title}}
{{title.lowercase}}
{{title.uppercase}}

Also, I've notice Mustache being used in front end scenarios where only JSON is allowed and therefore lambdas can't be used to accomplish this.

@thelucid

or...

var view = {
  title: {
    'default': 'Hello World',
    lowercase: 'hello world',
    uppercase: 'HELLO WORLD'
  }
};
{{title}}
{{title.lowercase}}
{{title.uppercase

Essentially, I'm saying we should be able to set a default value for an object and all the this comes under a "Coercion" umbrella. Super useful and super easy to implement.

@groue

It doesn't come from nowhere, it's a common pattern in most

Maybe. Still nobody asks this, and isn't needed by any solution to existing problems we are aware of. Just because we can do something does not mean we need to do it.

and gets around the whole filter debate for example

Absolutely not. For instance, one may need to format a single number as #.## here, and as a percentage elsewhere. A name as a raw value here, and the same name as uppercase somewhere else.

I've notice Mustache being used in front end scenarios where only JSON is allowed and therefore lambdas can't be used to accomplish this.

I've been thinking of that as well, wondering if we shoud strive for the Mustache spec to require consistent rendering, given a set of templates and input data, JSON being the preferred format for specify input data, since it is quite expressive, and a de facto intercommunication standard nowadays.

However, as soon as you let users inject code (lambdas, filters, some other hooks I'd love to talk about), one realizes the unity can't be acheived without processing of the input data before it is rendered. This processing can't be specified, since it depends on the actual API for code injection, and depends on the coverage of the SHOULD clauses by the implementation. Thus the spec will never be able to specify a consistent rendering, given a set of templates and input data, as soon as userland code is involved.

Plus, don't forget, to_string or whatever special keys must not enter the spec: the spec won't specify what should happen when a to_partial key is provided, since this key is the choice of a particular Mustache implementation, that chooses to use this particular key in order to implement Mustache automatic coercion. You do not want Ruby to be limited by Javascript, do you?

Essentially, I'm saying we should be able to set a default value for an object and all the this comes under a "Coercion" umbrella. Super useful and super easy to implement.

Feel free to implement it in your Mustache implementation. Nobody prevents you to do it.

@thelucid

Plus, don't forget, to_string or whatever special keys must not enter the spec

No, I agree, just using the JavaScript version for examples.

The "coerce to lambda" and "coerce to partial" would both be part of a "Coercion" spec, what I am saying is why not complete the spec by including the only other possible need for coercion which is "coerce to string". I would rate this higher than "coerce to lambda" personally. Funnily enough, I was going to create a separate ticket prior to this one for a way to specify a default value.

Let's say you have a CMS or a site like Shopify and you want to allow users to edit the templates, providing just JSON data, quite a common use case. You could document that the product price is available through {{product.price}} and the currency on it's own through {{product.price.currency}} (or any other property of the price) for example. You could even use it in a similar way to Ruby's inspect for example, to give a more expected/user friendly result than a blank string e.g. {{product}} could output [Product #123].

Feel free to implement it in your Mustache implementation. Nobody prevents you to do it.

I can't submit a pull request for something like this as it needs to be in the spec first. I could maintain my own fork, but why not allow others the functionality for what is probably only an additional line or two of code.

I think that "coerce to string" completes the "Coercion" spec nicely.

@bobthecow
Mustache member
var view = {
  title: {
    'default': 'Hello World',
    lowercase: 'hello world',
    uppercase: 'HELLO WORLD'
  }
};
{{title}}
{{title.lowercase}}
{{title.uppercase}}

Essentially, I'm saying we should be able to set a default value for an object and all the this comes under a "Coercion" umbrella. Super useful and super easy to implement.

I understand you're just trying to supply an example, but I'ma nitpick for a bit. You've already got this, built into your langauge:

var view = {
  title: 'Hello World'
};
{{title}}
{{title.toLowerCase}}
{{title.toUpperCase}}

... But even if your language didn't have it built in, it is trivial to add:

function cases(str) {
  var ret = function() { return str; };

  ret.uppercase = str.toUpperCase();
  ret.lowercase = str.toLowerCase();

  return ret;
}

var view = {
  title: cases('My Title Is Awesome')
}

And there's an analog for this sort of thing in almost every language. Languages which are missing it should feel free to implement a convention as a workaround. But there's no way a workaround for language deficiencies belongs in the spec :)

In my experience, most perceived deficiencies in Mustache come from a lack of understanding about the purpose of the ViewModel (rendering context, etc). That's where things like this belong. You should put code in there. If you're just passing a static hash of values, you're missing out of most of the power and expressiveness of Mustache.

But just because you're missing out doesn't mean the spec is deficient.

@groue

I'm currently trying to fit those ideas into GRMustache. Namely points 1 and 4 of the proposed spec update :

  1. An implementation MUST render of {{items}} the same way as {{#items}}{{.}}{{/items}} whenever the value attached to items is an enumeration.
  2. When rendering {{value}}, an implementation must fetch the value attached to the key value. If this value is a lambda, or can be automatically converted to a lambda, the rendering MUST be the result of the lambda call.
  3. An implementation SHOULD provide an API for letting the library user specify the automatic conversion to lambda described at point 2.
  4. An implementation SHOULD provide an API for letting the library user specify the automatic conversion to lambda described at point 2 via a partial name.

I expect some feedback from the implementation.

@thelucid

@bobthecow I guess Mustache isn't a good fit for a CMS or Shopify style app then as users could potentially call destructive methods on the view object. I've seen a lot of people try to have Mustache take the place of Liquid templates, would you say this is a bad idea in an environment where users can edit templates?

My uppercase/lowercase example was a bad one, granted. Can you see the benefit of an object to be able to give a sane default rather than an empty string? e.g. in Ruby "Time: #{Time.now}" results in "Time: Fri Sep 21 19:37:46 +0100 2012" rather than "Time: "

@bobthecow
Mustache member

@thelucid It's actually a perfect fit. You just need to use a proper ViewModel (which doesn't expose any methods you don't want called) rather than passing your data model directly to the template.

Yes, there's a benefit to sane defaults. Those sorts of things are super useful. And Mustache already supports them. In PHP, for example, you implement string coercion by adding __toString to your model, and lambda coercion by adding a __call method... Neither of these require an extension of the Mustache spec, because they're implementation details.

The responsibility of determining what's a proper default value belongs to the ViewModel. The responsibility for providing formatted versions of values belongs to the ViewModel. Adding semantic meaning to properties (default, to_string, to_lambda, whatever) is moving in the wrong direction.

@thelucid

@bobthecow I'm obviously having a blonde moment so bear with me. Why is __toString ok but not to_string? I'm not saying that all implementations should specifically have a to_string key but that they can "coerce to string" in some way that can be referenced in a template. In the case of JavaScript, this would be:

View

var data = {
  product: { id: 123, name: 'thing', toString: '[thing 123]' }
};

Template

Product: {{product}}

This would output: Product: [thing 123]

@bobthecow
Mustache member

__toString is a language feature. It's how, in PHP, you tell the interpreter that an object can be coerced into a string. The idiomatic way to do this in Ruby is completely different... All I'm saying is that this is an implementation detail, and doesn't belong in the spec.

@groue

@thelucid : what @bobthecow tells you is that string coercion is already right there, built in all existing Mustache implementations. PHP implementation uses default PHP coercion with __toString, and that is fine for all PHp developers. I guess the documentation of his implementation states here somewhere. Same for GRMustache: it uses the standard 'description' method.

Now you can write a JS Mustache implementation that says, in its documentation, that the special 'to_string' key would be used for string coercion instead of the default JS one. Nothing prevents you from doing that. @bobthecow suggests that it's not very useful, and that you miss the step of the view model preparation. I'd prefer say that the usefulness of this particular hypothetical implementation, and is destiny, belongs to the battlefield of JS Mustache impls, not to the spec domain.

Let me elaborate: I'm, personally, quite tired of hearing "just prepare your view model". Not every user exposes its templates à la Liquid. Some do not care if the templates have access to "forbidden" methods. Those are users who fill templates with actual models, not prepared view models, because preparing view models is a chore. This is my main argument for introducing filters in Mustache. Check https://github.com/groue/GRMustache/blob/master/Articles/WhyMustacheFilters.md. Generally, I wholeheartly support all kinds of help for those users, who just don't give a shit about view model nazis, and just want to have their damn template engine render their data.

So, @thelucid, there may be some room for your ideas, in order to help users quickly build their templates. The spec must not prevent it. But it must not force every implementors to do the same. Mustache can, and must, support both approaches.

@thelucid

Ahh, sorry both, I was pushing for something that already exists, the irony is, I didn't know it existed because it's not in the spec.

How do we move forward on the "coerce to lambda" and "coerce to partial key" stuff, should I implement this in the JS version and submit a pull request or do I wait for it to go into the spec first? Chicken or egg?

@groue

Go ahead and do it, since real artists ship :-)

You know, the spec shouldn't evolve that much, just recommending implementations to provide a way to provide automatic lambda coercion through a partial name, from "foo" to function() { return "{{> foo }}" }, on top of other coercions (that would benefit from being explicitly mentioned in the spec as soon as it is more than a pile of yaml tests).

Your main contribution here is the fact that a meaning for {{#items}} has emerged :-)

@groue

@thelucid your other main contribution, how could I forget it, was your initial idea of the special 'partial' key, that eventually led to a way to get rid of any new syntax, punctuation, dereference etc. I couldn't be too glad about this achievement :-)

@thelucid

Your main contribution here is the fact that a meaning for {{#items}} has emerged :-)

I'm guessing you meant {{items}}?

your other main contribution, how could I forget it, was your initial idea of the special 'partial' key, that eventually led to a way to get rid of any new syntax, punctuation, dereference etc.

Yes, glad we finally agreed on a way to do dynamic partials that we are both happy with.

@groue There is one problem with this approach, I believe that the contents of the partials get escaped. I'm not sure how to work around this.

@groue

@thelucid oops, sorry for the {{#typo}}.

I've finished implementing those ideas in GRMustache. They work so smoothly I'm impressed :-). I'm writing the doc right now and will ship GRMustache 5.1 right away.

I was concerned by the escaping of partials as well, wrote a test for it, and was prepared to see it fail. Actually, it wasn't :-), and here is why:

The code that renders variable tags has to check for the kind of object is is given. If it is nil/null/zilch/none/undefined/etc, no rendering happens. If it is a lambda that takes no parameter (I call them "variable lambdas"), it renders it, and insert the rendering right away, without escaping. In any other cases, it applies string coercion, and HTML escaping if needed. Escaping does apply for lambdas. Both of us forgot this point :-)


Just to be clear, here is the ideas I'm implementing in GRMustache:

  1. Considering that dynamic partials can already be implemented with lambdas, that is to say at the view model level,
  2. the spec should embrace this fact, and should not evolve in a way that alter the Mustache language just for the sake of dynamic partials. No new punctuation, no new concept as dereferencing.
  3. The spec should state that implementors SHOULD provide a handy API for building lambdas that render dynamic partials.
  4. Furthermore, inspired by Rails' <%= render @items %>, the spec would benefit from specifying that {{items}} should be rendered as {{#items}}{{.}}{{/items}}.
@thelucid

@groue Good work.

I'm currently struggling with making the JS version render {{items}} as {{#items}}{{.}}{{/items}}.

Also, I don't see that this solution is possible in the current JS implementation as function() { return '{{>image}}'; } just outputs {{&gt;image}}, not the contents of the partial.

@groue

@thelucid It looks that the JS implementation that you base your work on misses the lambda feature :-( What about switching to Objective-C, because you could right away render {{items}} just as <%= render @items %> :-)

GRMustache 5.1 is out, with extended variable lambda support, dynamic partial support, and collection rendering.

The documentation of lambdas and dynamic partials is at https://github.com/groue/GRMustache/blob/master/Guides/helpers.md. It's a thorough doc, and some of you may not be used to Objective-C verbosity. Please focus on the three sections whose title starts with "Variable helper example". I believe a Mustache implementation should make sure it supports the three of them.

The last example is the most striking:

Data:

base.mustache
{{movie}}

movie.mustache
{{title}} by {{director}}

person.mustache
{{firstName}} {{lastName}}

... rendering:

Citizen Kane by Orson Welles

... all without much view model preparation. Something like a to_partial key - but in Objective-C clothes.

@thelucid

It looks that the JS implementation that you base your work on misses the lambda feature :-(

It has lambda support but not returning Mustache from a lambda, is this even part of the spec anyway? Guess I'm back to square 1. Dynamic partials not possible?

What about switching to Objective-C, because you could right away render {{items}} just as <%= render @items %> :-)

Objective-C is not really a solution for web development.

GRMustache 5.1 is out, with extended variable lambda support, dynamic partial support, and collection rendering.

Nice.

@groue

It looks that the JS implementation that you base your work on misses the lambda feature :-(

It has lambda support but not returning Mustache from a lambda, is this even part of the spec anyway? Guess I'm back to square 1. Dynamic partials not possible?

Yes it is in the spec, but @pvande who wrote it never managed to let implementors know. So many implementations base their code on the lambdas described at http://mustache.github.com/mustache.5.html, which is not spec-compliant. :-) Ain't it hilarious?

However, you should have a render function available somewhere, don't you?

(BTW, the spec is cumbersome, and I never implemented it as it is. I allow my users to write spec-compliant code if they need it, but I also let them render their own crafted text if they prefer - see last paragraph of https://github.com/groue/GRMustache/blob/master/Guides/helpers.md, "GRMustache helpers vs. Mustache lambdas").

@thelucid

Since the spec doesn't mention that lambdas should be able to return unrendered Mustache and have it rendered. We will need to change this statement:

The spec should state that implementors SHOULD provide a handy API for building lambdas that render dynamic partials.

...to:

The spec should state that implementors SHOULD provide an API for rendering dynamic partials via a partial key.

@groue

Since the spec doesn't mention that lambdas should be able to return unrendered Mustache and have it rendered

Actually, does it mention anything but a bunch of YAML tests?

@groue

Two gists illustrating the relationship between lambdas and dynamic partials :

A gist illustrating that GRMustache can render {{ item }} and {{ items }} as Rails does <%= render @item %> and <%= render @items %>:

@thelucid

Actually, does it mention anything but a bunch of YAML tests?

Don't quite follow. I thought your original argument against my implementation was that it doesn't fall inline with the spec (which is fair enough), yet returning mustache from a lambda and expecting it to get parsed is also not defined in the spec, you can't have it both ways ;)

Surely having the lambdas parse mustache is a bad idea as you end up in an n+1 situation. For every lambda, another mustache string is parsed, and that's even worse for a collection of lambdas. Surely this is going to have a performance impact on a view with many lambdas or collections of lambdas?

I always assumed that lambdas were expected to return a plain string that would then be escaped, hence my not getting your first example that returned {{>image}}. Turns out that this is how pretty much all implementations behave (I believe), except for of course the Objective-C one.

In other implementations you just use string concatenation if you want to achieve the same {{link}} functionality as your gist. Far better for performance overall.

So we need to agree on a spec that is possible in all implementations given the current spec which I believe to be:

  1. An implementation MUST render {{items}} in the same way as {{#items}}{{.}}{{/items}}, whenever the value attached to items is an enumeration.
  2. An implementation SHOULD provide an API that allows objects to render themselves via a specified partial.
@groue

I always assumed that lambdas were expected to return a plain string that would then be escaped, hence my not getting your first example that returned {{>image}}. Turns out that this is how pretty much all implementations behave (I believe), except for of course the Objective-C one.

Actually I just implemented lambda this morning. It looks like nobody in the Obj-C world (which is not that much web-oriented) was missing them before. But dynamic partial were such a needed features, and lambdas such a good host for them :)

I think Python and Ruby implementations, at least, follow the spec, and interpolate the lambda output. Maybe some others do.

Actually, this is useful. Think of it as an inline dynamic partial.

Surely this is going to have a performance impact on a view with many lambdas or collections of lambdas?

A nice Mustache implementor could cache the parsing, so it's not such a burden on the CPU.

In other implementations you just use string concatenation if you want to achieve the same {{link}} functionality as your gist. Far better for performance overall.

Yes, you are perfectly right: the only flaw in the current spec is that it requires the interpolation of the lambda output.

So we need to agree on a spec that is possible in all implementations given the current spec which I believe to be:

Looks legit to me.

@groue

Quoting myself:

the only flaw in the current spec is that it requires the interpolation of the lambda output.

You know, an implementation can still have another behavior, and allow both spec-compliant interpolation and raw rendering. It just has to state it very clearly in its documentation: "If you need interoperability with other Mustache implementations, you need to use the interpolation API, and avoid the raw rendering."

That's what GRMustache has done from the beginning, when I did not agree with the spec.

@groue

Meaning: you can alter mustache.js the way you want. You don't have to wait for the spec to evolve.

@groue

@thelucid Oops, I made a big mistake. Remember when you said:

There is one problem with this approach, I believe that the contents of the partials get escaped. I'm not sure how to work around this.

You were right. I was fool not escaping lambdas. Hold on.

@groue

I was fool not escaping lambdas. Hold on

Nope. It's OK. If a user wants escaping, he uses the interpolation mechanism. If he does not want escaping, he returns a raw string. Phew.

@thelucid

So, to cut to the chase... how do I implement this in mustache.js when we don't have a clear spec (apparently mustache.js gets lambdas wrong) and how do we get this in the spec so that all implementations can benefit??

@thelucid

Does anybody else have an opinion on this?

@groue

So, to cut to the chase... how do I implement this in mustache.js when we don't have a clear spec (apparently mustache.js gets lambdas wrong) and how do we get this in the spec so that all implementations can benefit??

@thelucid As I said before, you can extend mustache.js right now. Just make sure to provide a spec-compliant facet of your API, and another facet for the features that the spec is missing.

@devinrhode2 @thelucid When I look at most open issues in this repo and in main (most-followed/forked) implementations, it looks that GRMustache have found a solution for nearly all of them, but template inheritance (#38) and masked data (#11).

Template inheritance has already been implemented by twitter/hogan.js and spullara/mustache.java, so it would be wise consider it a closed case, and base a spec evolution on their implementations.

Masked data simply does not motivate me enough, so I plan to ignore it.

So I plan to write my own version of the spec, making sure not to introduce any incompatibility with Mustache 1, in order to avoid a "Mustache 2" naming that may scare existing implementors. It will be Mustache 1.2 and I hope it will be supported by other implementors. Let me implement template inheritance first, and we'll have a full-featured implementation, with documentation and sample code, that will show the usefulness of the spec evolution.

@groue

Template inheritance, aka overridable partials, done. Documentation plus example.

@thelucid

There are a couple of issues that are stopping me from implementing Ruby and/or Javascript versions:

  1. All implementations currently escape strings returned from lambdas making the dynamic partial solution that you have suggested impossible in any implementation other than your own.
  2. Neither mustache.js or hogan.js render the strings returned from lambdas as mustache fragments and even if they did, the suggested solution is still not possible due to 1.

Based on this, the only way forward (that I can see) is to amend our criteria to what I had suggested previously (slightly reworded):

  1. An implementation MUST render {{items}} in the same way as {{#items}}{{.}}{{/items}} when the value attached to items is an enumeration.
  2. An implementation SHOULD provide an API that allows an object to specify the name of a partial to automatically render with (dynamic partials).

If we can agree on this, I can move forward on an implementation.

@groue

@thelucid I see your points now, and they are quite strong. The points 1 and 2 look sane. Point 1 looks almost obvious and won't surprise anybody (and we should add {{{items}}} <=> {{#items}}{{{.}}}{{/items}} for completeness). Point 2 is based on an API, so we'll still have room for other APIs as we see them emerge. Why haven't you started your implementation yet? :-) I'm still looking how to avoid the interpolation of lambdas output that the spec requires today.

@groue

@thelucid

Just for the record, here is how GRMustache today avoids the issue with lambda escaping:

  1. output of lambdas is directly inserted in the final rendering, without any escaping, regardless of the number of braces in the template.
  2. they are provided with an API to render a custom template string (so that you can write spec-compliant lambdas).
  3. they are provided with an API to render a partial by name.
  4. lambdas are the top object of the context stack when they are rendered, so that they can use those APIs in order to render HTML-escaped custom values.

Translated in a hypothetical JS implementation, this would give:

Example: rule 1

var lambda1 = function(M) {
    return "<unescaped>";
}

// "<unescaped>"
render(
  "{{lambda1}}",
  { lambda1: function() { return lambda1; } });

Example: rules 1, 2, 4

var lambda2 = function(M) {
    return M.render("{{text}}");
}
lambda2.text = "<escaped>";

// "&lt;escaped&gt;"
render(
  "{{lambda2}}",
  { lambda2: function() { return lambda2; } });

Example: rules 1, 3, 4

var lambda3 = function(M) {
    return M.render_partial("partial");
}
lambda3.text = "<escaped>";

// "&lt;escaped&gt;"
render(
  "{{lambda3}}",
  { lambda3: function() { return lambda3; } }),
  { partial: "{{text}}" });

I state that this behavior is not anti-spec, since one can implement spec-compliant lambdas given this set of rules and APIs. Yet it's more versatile.

I agree that it's not easy for an existing implementation to switch to this behavior.

So I don't want to sound assertive :-) What do you think?

@groue

(fixed typos in previous comment)

@groue

You would be sad if I would not provide an example with the to_partial key :-)

// "&lt;escaped&gt;"
render(
  "{{lambda4}}",
  { lambda4: {
      to_partial: "partial",
      text: "<escaped>" }
  },
  { partial: "{{text}}" });
@groue

Looking at @janl's mustache.js. This is fascinating.

The following tests pass:

higher_order_variable.js
({
  text: ">",
  a: "<{{text}}>",
  b: function() {
       return "<{{text}}" + this.text;
     },
  c: function() {
       return function () {
         return "<{{text}}" + this.text;
       };
     }
})

higher_order_variable.mustache
{{a}}
{{b}}
{{c}}

higher_order_variable.txt
&lt;{{text}}&gt;
&lt;{{text}}&gt;
&lt;{{text}}&gt;

So clearly, mustache.js provides no support whatsoever for Mustache lambdas as defined by the spec, that is: lambdas that can render a custom template string in the current rendering context.

So. What about using the c form in order to implement our modern-style lambdas in mustache.js?

  • The c form does not add anything to the b form, so we can assume nobody uses it.
  • There is no test for the c form in mustache.js, so we can assume we are free to do whatever we want with it.
  • The c form is very close to the "higher order sections" of mustache.js, tested in test/_files/higher_order_section.js

So your have room, @thelucid, to implement variable lambdas with the c form. For instance, keeping a and b intact, modify the c handling form so that:

higher_order_variable.js
({
  text: ">",
  a: "<{{text}}>",
  b: function() {
       return "<{{text}}" + this.text;
     },
  c: function() {
       return function (render) {
         return render("<{{text}}")+this.text;
       };
     }
})

higher_order_variable.mustache
{{a}}
{{b}}
{{c}}

higher_order_variable.txt
&lt;{{text}}&gt;
&lt;{{text}}&gt;
<&gt;>
@thelucid

I think it's generally a bad idea to not have the results of lambdas escaped, it's open to human error just like Rails was until auto-escaping was introduced in views. I tend to think that the other implementations have this behaviour right.

So your have room, @thelucid, to implement variable lambdas with the c form. For instance, keeping a and b intact, modify the c handling form

This also seems like a bad idea to me, as (other than the aforementioned) it will add an inconsistency to the API.

Why haven't you started your implementation yet? :-)

I've started looking into some options but wanted to agree on a spec first.

I really do think that the only way forward (that fits with existing implementations and spec) is the simplified:

  1. An implementation MUST render {{items}} in the same way as {{#items}}{{.}}{{/items}} when the value attached to items is an enumeration.
  2. An implementation SHOULD provide an API that allows an object to specify the name of a partial to automatically render with (dynamic partials).

Don't you think?

@groue

@thelucid

The problem with your focus on dynamic partials is that you forget the really nice contribution of the spec, which requires interpolation of the lambda output. The spec does a mistake requiring it. However it is quite useful to provide users a way to render/interpolate a custom template string. Mustache.js does not implement it today, for some reason I don't care about. Please at least try to include this feature in your proposal. As I said before, you may think of it as an inline partial (render @item is just a particular case of render partial:. render inline: is just as useful.)

I think it's generally a bad idea to not have the results of lambdas escaped, it's open to human error just like Rails was until auto-escaping was introduced in views. I tend to think that the other implementations have this behaviour right.

I disagree, or to be more precise, please reconsider. The forms a and b have their result safely escaped, and are already quite versatile. Very few users would need to use the more "dangerous" form c, just as very few users need to use the raw/html_safe methods of Ror.

This also seems like a bad idea to me, as (other than the aforementioned) it will add an inconsistency to the API.

Your ideas are not clear enough, and you jump on wrong conclusions.

What I propose you is to have mustache.js evolve in 1. a backward-compatible way with itself, 2. a way compatible with its initial total ignorance of the spec regarding lambdas, 3. a way compatible with the mustache spec, 4. a way that allow users to render what they want, including in a spec-compliant way.

Let me prove it to you:

  1. because the form c is not specified by the mustache.js tests, and not used by any user. So you can do whatever you want with it, and nobody will be affected.
  2. By focusing your changes of form c, the form b will not be changed, that "lambda of the poor man".
  3. By using M.render("{{..}}") in the form c, users can emulate a spec-compliant lambda.
  4. The form c allows any kind of rendering, and I love to free users from absurd constraint.
@thelucid

I agree that interpolation of lambda output can be handy in some cases (although a bit of a hit on performance). It's not something I've ever needed but others may. As you say, this can be emulated with M.render("{{..}}") arguably a better solution as there is only the extra overhead when the feature is needed.

What I am saying is that I think all other implementations are correct in always escaping lambda output when using {{...}}, and not escaping when using {{{...}}}, it's the expected behaviour that the double mustache escapes values and the triple does not. Are you saying your implementation is correct and all others are wrong?

I don't see that there needs to be a "dangerous" version at all if we just change our expectations for dynamic partials to my two step alternative. If you do need to use "dangerous" lambdas then be explicit that they are dangerous by using the triple mustache, this also illustrates to others that they may be editing a given template that there may be unescaped content coming from the view. Template authors should have the confidence that when they use the double mustache, the content will be escaped regardless of how the view is implemented.

@groue

I agree that interpolation of lambda output can be handy in some cases (although a bit of a hit on performance). It's not something I've ever needed but others may. As you say, this can be emulated with M.render("{{..}}") arguably a better solution as there is only the extra overhead when the feature is needed.

Cool

What I am saying is that I think all other implementations are correct in always escaping lambda output when using {{...}}, and not escaping when using {{{...}}}, it's the expected behaviour that the double mustache escapes values and the triple does not. Are you saying your implementation is correct and all others are wrong?

I say that nobody implement lambdas as specified by the spec, because they are insane, and that GRMustache is the only sane implementation I know about. Look at the following tests:

Those tests are crazy, because they lead to double escaping, which is not better than no escaping at all, and makes lambda nearly useless. According to the spec:

// Some template we share with other people and that we can not change.
var text = "{{person}}";

// 1st rendering: {{person}} renders as a person name:
//
// &lt;pwnd&gt;
render(
    text,
    { person: "<pwnd>" }    // nasty user data :-)
)

// Actually, we'd better render a link. Let's use a lambda, so that
// we use Mustache's HTML escaping :-)
//
// &lt;a href=&quot;http://...&quot;>&amp;lt;pwnd&amp;gt;&lt;/a&gt;
render(
    text,
    { name: "<pwnd>",
      url:"http://...",
      person: function () { return '<a href="{{url}}">{{name}}</a>'; }
)

Of course this is ridiculous. The only way to avoid this problem would be to use triple mustache, which means changing the template. And all other projects which use the same template will have to update now.

This is a dramatic flaw of the spec.

Now. Let's consider real Mustache implementations. They can fill three sets:

  1. implementations that do not support lambdas for variable {{tags}} (mustache.js and many others)
  2. implementations that implement lambdas for variable {{tags}} as required by the spec (lambda returns a template string that Mustache interpolates, and then escapes)
  3. implementations that implement lambdas for variable {{tags}} in a way that allows users to opt in the "fixed spec behavior", that is to say without the crazy double-escaping (GRMustache).

It should be very clear in your mind that mustache.js as no support for variable lambda at all. The form b is a convenience for allowing computed properties to be rendered, for objects like:

{ firstName: "Gwendal",
  lastName: "Roué",
  fullName: function() { return this.firstName + " " + this.lastName; } }

I state that implementations of the fist group are free to implement whatever they want, as long as it is sane. Mustache.js can.

Implementations of the second group have chosen the wrong way, I'm really sorry for them.

And GRMustache is the only one I'm aware of that can handle the above scenario, variable lambdas, dynamic partials, and the {{items}} syntax in a sane way.

@thelucid

I guess we should get the opinions of @janl and @defunkt on your suggestion of allowing unescaped content within double mustaches. I personally would like to keep double mustaches safe, regardless of the view code.

Perhaps that issue deserves a new ticket. I say we concentrate on coming up with a solution for dynamic that meets the current spec and that is possible in current implementations i.e.

  1. An implementation MUST render {{items}} in the same way as {{#items}}{{.}}{{/items}} when the value attached to items is an enumeration.
  2. An implementation SHOULD provide an API that allows an object to specify the name of a partial to automatically render with (dynamic partials).
@groue

I guess we should get the opinions of @janl and @defunkt on your suggestion of allowing unescaped content within double mustaches.

Me too

I personally would like to keep double mustaches safe, regardless of the view code.

I think I've clearly shown with the {{person}} example above how it escaping of lambdas totally ruins the ability of a lambda to override a simple property. And remember how the b form of mustache.js is not a lambda (as in fullName: function() { return this.firstName + " " + this.lastName; }. It's a mere convenience for computed properties. A "real" lambda for mustache.js should use double function as in mutache.js's test/_files/higher_order_section.js : fullName: function() { return function(M) { return M.render("{{firstName}} {{lastName}}"); } }

And obviously in this case you do not want double escaping, and be forced to switch from {{fullName}} to {{{fullName}}}.

Would you implement the ability of {{item}} to render the partial item.mustache unescaped, and disallow {{item}} to render the rendering of {{>item}} unescaped ? Where is the consistency?

@groue

I mean: do what you believe is best. Your two rules are OK. And I understand why you want to implement partial coercion without using lambdas. Go ahead :-)

@groue

Whenever @defunkt, @janl, @pvande will want to have a look, the GRMustache documentation on lambdas for variable tags is ready: https://github.com/groue/GRMustache/blob/master/Guides/variable_helpers.md. A few example gists: https://gist.github.com/3765844, https://gist.github.com/3765846, https://gist.github.com/3765850

@groue

I've updated my lambda documentation, with a section about compatibility with other implementations.

@groue

Dynamic partials need special care when templates are stored in a hierarchy of directories.

GRMustache has just shipped with support for both relative and absolute paths to partials.

Regular partial tags contain relative paths: {{> header }}, {{> partials/header }}.

Absolute paths start with a slash: {{> /path/to/partial }}.

Release notes, and link to documentation: https://github.com/groue/GRMustache/blob/master/RELEASE_NOTES.md#v540

@thelucid

Apologies for not replying until now, I've been otherwise engaged. I hope to have a crack at the Ruby and Javascript implementations shortly. What do you think the likelihood is of having the changes accepted?

@thelucid

As a side-note, I'm thinking of creating a "Safe Mustache" implementation or even a "safe" configuration option in Mustache itself.

I was looking at using Mustache in a similar way to Shopify uses Liquid i.e. allowing end users to edit templates directly. This is not a good fit currently as even though (as pointed out my @bobthecow) you could remove methods that you don't want users to call (kind of like Ruby's BlankSlate object), users will always be able to get to the underlying language methods on strings such as inspect etc.

I envisage a "safe mode" much like how Liquid works i.e. only the keys and value that are explicitly exposed in the view object are allowed. Something like {{title.downcase}} or {{collection.inspect}} (Ruby example) should just return empty strings unless they are explicitly defined.

What do you think? ...probably deserves a separate ticket.

@devinrhode2

@thelucid, sounds like an implementation detail, but I'd agree that there's a variety of properties and objects that one does not want available to the mustache template.

@groue

@thelucid, @devinrhode2 About safe templates: @pvande was already choked by them (see #41 (comment)). Yes it needs another ticket. Beware losing your time, since it's heavily implementation-dependent. Plus the Liquid use-case (rendering untrusted templates in a safe VM) is, frankly, not common at all. If you work on this, please make sure the safe mode is optional. Nothing is more frustrating than a software that prevents you to hack your solutions, for the sake of some "safety" you do not give a shit about when you do not run untrusted templates. Make the users free and responsible, do not nanny them: https://github.com/groue/GRMustache/blob/master/Articles/TheNatureOfLogicLessTemplates.md

@thelucid

My take on a 'safe' Mustache implementation:

Next job, dynamic partials.

@thelucid

@groue I've been looking at options regarding the rendering of lambda output in double taches. How do you feel about introducing the idea of a safe string in a similar vein to Rails? This would solve all of our problems with the following rule:

  • All output should be escaped by double taches {{...}} unless type is "safe string".
@thelucid

@groue One other thought regarding the 'blessed key' problem you highlighted with dynamic partials, we could introduce a > key that allows you to specify a partial to render with. This wouldn't break backwards compatibility as that symbol is not allowed in tag names.

Also . could be used to define a default value (a shortcut for to string). These shortcuts would then be usable with hashes rather than needing objects.

@groue

How do you feel about introducing the idea of a safe string in a similar vein to Rails?

I think this is the only viable option, as soon as you want to let the user write non trivial lambdas.

Handlebars exposes Handlebars.SafeString to its users.

GRMustache, while remaining a Mustache engine, asks its users to set a HTMLSafe flag (doc).

I'm happy you are beginning to explore this fruitful path.

@groue

dynamic partials ... > key
. ... a shortcut for to string

I believe your work on safe strings will enlighten you, just as I have been.

"Dynamic partials", "string coercion" are nothing but particular cases of interaction of an object with a Mustache tag. There is no need for specific APIs for any of those cases.

A generic API that lets the user provide its own tag-interaction code is just enough. I don't care about the shape of this API. Use a special key, whatever, do with what Javascript can do.

What I care about is the unicity of the interaction point. For an illustration, see again GRMustache rendering API. Don't miss the sample codes at the bottom: you'll see which kind of bad-ass job can be done.

@thelucid

Ah, I didn't notice before that you had implanted a safe string, I really think this is all that is needed and should enter the spec i.e. safe-strings.yml:

  • All output must be escaped by double taches {{...}} unless type is "safe string".

This solves everything, even dynamic partials and self rendering objects. I am just working on my SafeString implementation... this in combination with 'safe views' for end user templates is going to be a really powerful combination. How do you feel about an optional 'safe view' module entering the spec i.e. ~safe-mode.yml?

@thelucid

On second thoughts, the first rule should be part of interpolation.yml and the second just ~safe.yml.

@groue

you had implanted a safe string

Not at all like the Rails' one, which support operators, concatenation, and generally speaking, carries its "safe" state.

GRMustache has a HTMLSafe flag that allows the user to say if a string should be interpolated or not.

Handlebars has something identical, but with a different API: users wrap their result string in a Handlebars.SafeString object.

The important thing, in Handlebars and GRMustache, is to let the user provide his own code when he needs to, and to let him decide whether his carefully crafted strings should be interpolated or not.

Both libraries have a "safe string" concept that is used in their callback API, when the Handlebars/Mustache engine calls back user's code that renders a tag. Handlebars limits the callback to its "helper" concept. GRMustache generalizes the callback to all tag (variable + section) renderings.

In Handlebars and GRMustache, there is no such thing as a "safe string" that would enter the view model. You can search for "Handlebars.SafeString" in http://handlebarsjs.com, and "HTMLSafe" in https://github.com/groue/GRMustache/blob/master/Guides/rendering_objects.md. If you have time. If you are interested.

@groue

BTW. I consider the topic of "dynamic partials" as 100% done in GRMustache. They are implemented as a particular case of the callbacks I'm telling about, above.

Let me tell you one thing. Dynamic partials are a nice feature. Not very often requested by users, compared to two major drawbacks of the Mustache spec: its painful focus on "view model preparation" which make basic stuff such as formatting numbers or rendering array indexes a chore.

Now all repos have issues about those two particular topics.

I consider the primary job of a Mustache engine implementor to be able to solve those issues. Both require a versatile API. Formatting can not be hard-coded in the engine: there are two many ways to format data. Array indexes can not be hard-coded in the engine: some count from 0, some from 1, some want to play fizz buzz.

The GRMustache engine provides APIs that, as a side effect, also support dynamic partials. That is because they are versatile.

I do not believe in adding one tiny feature after the other. I believe in exposing a few powerful callbacks and hooks so that he can render what he needs to render with his tool. Mustache is a mere tool, not an opinionated framework. Handlebars got this important idea very early, and forked from the "pure" Mustache so far that now it's a different engine. I strived, in GRMustache, to bring as much versatility, if not more, while remaining a Mustache engine. That job is done, today. GRMustache is done. A Mustache engine that does not suck, and lets you get the shit done. I'm just telling you the ingredients of this story.

@thelucid

Still investigating... One other thing I'm looking into is allowing a compiled template to be returned from a view method or lambda, if returned, it is rendered with the correct indent level. So basically there are three types of objects that can be returned, a string, a safe string or a compiled template. Initial results are good.

@groue

Great, Jamie :-) Glad you're investigating as well :-)

@dasilvacontin

It was interesting reading this. Almost two years later, heh.

@spiffytech

bump

Any movement on a v2 spec that includes this feature?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment