Skip to content
This repository

formating of dates, numbers and more #41

Open
rasmusfl0e opened this Issue March 19, 2012 · 91 comments

9 participants

Rasmus Fløe Jan Lehnardt Gwendal Roué David Santiago Anders Hellerup Madsen Justin Hileman Sam Pullara Pieter van de Bruggen Hendy Irawan
Rasmus Fløe

I've been discussing the viability of mustache as a templating solution with a colleague and he raised some questions about a lacking key feature; formating. And I tend to agree with him - having this as part of the mustache spec would be very helpful.

The string representation of a date or time, the numbers of decimals of a price or the number of characters shown shouldn't be part of your viewmodel - it should (in a perfect world) be handled by the templating language.

I'm thinking something along the lines of being able to define a "format" in some form immediately after the variable name:

{
    price: 2.7532
}

{{price:0.00}}USD

2.75USD

I know JSON doesn't support a native date representation - maybe just treat an ISO datestring as a date object and present it through a format like php's date function (other formats might be more fitting/standard)?

I'm sorry if this issue has been raised a thousand times (it seems so obvious somehow) - but I haven't been able to find an answer to this yet ;D

Jan Lehnardt
Owner
janl commented March 19, 2012

Would this work for you?

{
price: 27532,
price_format: function() {
// implement format_price();
return format_price(this.price);
}
}

and: {{format_price}}

PS: don't use floats for decimals, especially money.

Rasmus Fløe

That would mean I would "pollute" the viewmodel with a lambda - of which I'm not too keen... :)

And if I'm using the same templates in multiple languages across server/clientside I would also have to have implementations of these lambdas in those too (and maintain them). Then the lustre of mustache quickly wears off :(

Mustache could be the silver bullet :)

Floats/decimals was just a bad example :D

Jan Lehnardt
Owner
janl commented March 19, 2012

I like the idea of helpers, hence my mentioning them in http://writing.jan.io/mustache-2.0.html and my "helpers" branch here: https://github.com/janl/mustache.js/tree/helpers a helper would implement your number format, but you'd still implement them server-side and client-side.

fwiw, polluting view objects with lambdas is rather cool I came to appreciate even though it feels dirty.

Rasmus Fløe

Ah, I see.

I like the separation that helpers provides - definately more "separation of concerns" than lambdas ;D

Is it going in the spec anytime soon?

Gwendal Roué
groue commented March 19, 2012

This request is so common I had to provide with a solution for my Objective-C implementation. Three goals : no alteration of the Mustache syntax itself, no cooperation ("pollution") of the viewmodel, and provide an expressive-enough API so that users can hook in missing features we haven't thought of yet. https://github.com/groue/GRMustache/blob/master/Guides/sample_code/number_formatting.md

TL;DR : by providing carefully crafted hooks in the Mustache rendering engine, we can let users extend Mustache in many ways, and keep the "kernel" clean and simple.

Jan Lehnardt
Owner
janl commented March 19, 2012

basically what @groue says, all we need to get is an update to the spec and convince the people involved with it :)

David Santiago

Just for discussion, two other ways I've seen of doing this are in ctemplate's template modifiers and Liquid's filters. They're both pretty similar to each other.

Gwendal Roué
groue commented March 19, 2012

A clarification : as I said, my goal was to provide a solution which does not alter the Mustache syntax. My goal was to provide an expressive Mustache API. The spec and the syntax, as they are today, did not block me.

I think we should not focus on the syntax. For me, it cannot be the right level, since there will always be some users asking for missing features, and the syntax will always be late.

Hooks are the right level. Handlebars, on this topic, is particularly brilliant.

Gwendal Roué
groue commented March 19, 2012

A simple example of the Handlebars expressivity : https://gist.github.com/1048968

Rasmus Fløe

Yeah - spec could bloat if features get added without merit... I like the idea of helpers/hooks.

You could pick and choose the helpers - and hopefully write/port them as you need them for other languages.

I like the syntax of Liquid's filters though (not too dissimilar to janl's helper syntax). Imagine if you wrote a generic "date" helper that could get a format argument like so:

{{myDate|date:"%Y %h"}}

Obviously myDate would be an ISO datestring :)

Anders Hellerup Madsen
ahem commented March 20, 2012

I really like the idea of helpers as well. I feel that lambdas introduce logic into the viewmodel, and IMO there isn't really any difference between logic in the template and logic in the viewmodel - neither of which are all that desirable.

The filters or modifiers from ctemplate or liquid, as @janl suggests above looks very interesting.

@groue's handlebars way might be a good solution as well, but is the best tag really {{#somthing }}? I mean #-sections already have way too much functionality in them (if-statements, loops, scoping and so on). Also, it would mask the helpername so that you could no longer access properties with that name, wouldn't it?

Gwendal Roué
groue commented March 20, 2012

@groue handlebars way might be a good solution as well, but is the best tag really {{#somthing }}? I mean #-sections already have way too much functionality in them (if-statements, loops, scoping and so on).

I don't know if Mustache should jump in the Handlebars train. Yet we should consider its design with interest. Many mustache/spec issues do not exist in Handlebars, because Handlebars is expressive enough to let developers find a solution to their problems without asking for a language patch. And this really should ring a bell for the Mustache designers.

Its key design decision was to push the rendering out of the "kernel", and put it in "userland", with built-in implementations for the most common cases (bools, loops, scoping, mainly). Since rendering is done in userland, Handlebars users can use the public Handlebars APIs in order to build their own extensions to the language. They are just as powerful as the built-in helpers, which use public APIs as well. This really was a smart move.

Mustache does not totally prevent this expressivity, but unfortunately the only hooks that the Mustache spec provides are lambdas, which are so weak they can not help implementing basic stuff like number formatting or array indices, for instance. As a consequence, because we live in a world that favors pragmatic solutions over theoretical purity, Mustache implementors give lambdas extra powers, out of the specification guidelines, or even provide other hooks, not even thought by the current specification (my GRMustache, maybe others, is in this case), or, worse, play with the syntax (and that's really not smart, since compatibility is key to Mustache).

My opinion is that we should externalize in userland some Mustache features as well, and really avoid touching the syntax too much. Even if sections gain even more responsabilities. Finding unambiguous and non-collapsing names for "active" sections and viewmodel properties should not be too hard.

Just for instance, GRMustache hooks let you do the following rendering, without any built-in support, whatsoever, for number formatting:

raw: {{float}}
{{#percent_format}}percent: {{float}}{{/percent_format}}
{{#decimal_format}}decimal: {{float}}{{/decimal_format}}

gives:

raw: 0.5
percent: 50 %
decimal: 0,5

Everything is done in userland, in the code of the GRMustache user.

In conclusion: first start with userland, check the expressivity of your API, check if you can do anything useful with it. And then, after the API has been carefully tested, that its expressivity has been proven, then maybe, think about providing a Mustache standard library of helpers/hooks, like ctemplate or liquid. Thinking about this standard lib right now is premature.

Anders Hellerup Madsen
ahem commented March 20, 2012

I agree that this should be done in userland. All the spec should do is provide the hooks to make it possible to provide this form of extensions. I still stand by my objection to adding more features to sections, however. There is already several open issues here regarding that (#14, #22, #23 and others) so I am not the only one who feels like this. Also there is this:

<time datetime="{{#format_isodate}}{{myDate}}{{/format_iso}}">{{#format_readable_date}}{{myDate}}{{/format_readable_date}}/time>

It may just me, but I do feel that this is more readable (even with the strange PHP formatting letters):

<time datetime="{{myDate|date:"%c"}}">{{myDate|date:"%j. %F %Y"}}</time>

The arguments in the proposed filter syntax seems to be somewhat of a necessity, if the hook should be useful for formatters. Writing a number of decimal formatting tags like {{#2_decimals_format}} {{#3_decimals_format}} {{#4_decimals_format}} and so on seems slightly silly to me.

Gwendal Roué
groue commented March 20, 2012

I personally feel that literal number and date formats (%j, %Y and friends), right in the template, are smelly: template localization is going to be painful, and not everybody speaks English. Literal formats are not a necessity, and should even be avoided.

I'd rather have indirect formats {{myDate|date:iso8601}}, {{myDate|date:short}} {{myDate|date:long}}.

And now I can't see why {{#format_iso8601date}}{{myDate}}{{/format_iso8601date}} is really less readable, even if it looks more verbose.

Moreover, sections allow for formatting not only a single number, but all numbers in a whole section of the template. Think: ISO date across a whole XML document, with a single section. Think: currency format across the whole invoice HTML document, again with a single section.

Justin Hileman

How 'bout

{{#date.iso8601}}{{myDate}}{{/date.iso8601}}
{{#format.iso8601date}}{{myDate}}{{/format.iso8601date}}

? Then you can have hashes full of formatter lambdas, and indirect formats rather than literal formats.

Rasmus Fløe

I'm sold on the indirect formats - much better than the hard-to-read php/ruby formating :)

Anders Hellerup Madsen
ahem commented March 20, 2012

Okay, so the PHP dateformatting example was a poor one, you're right in that literal date formats should not be used. But wouldn't implentation be a problem with the section based approach? Maybe this is too early to discuss actual implemention, but I do feel that it should at least be considered.

{{myDate|format_iso8601date}} is easy, it maps to something like registeredHooks.format_iso8601date(value);. This is easy to implement and easy to understand.

With sections it's possible that more formatters could be active at one time, so some kind of protocol is required to select which formatter will be used. In GRMustache you have selected a pretty simple protocol - only numbers are formatted so the formatters only have to be called if what is outputted is a number, and you only have to call the most recently enabled formatter.

If this were a more generic setup, where hooks could be used to format anything, then either each hook would need to provide a callback where it could opt in or out, as to whether it would like to format a given value, or you would need some other way to decide.

Maybe all formatters are called in the order they were enabled, starting with the most recently enabled one, and then they would have to return either the original value, or a formatted version of it.This would mean that a template like this: {{#format_numbers}}{{#format_dates}}{{myDate}}{{/format_dates}}{{/format_numbers}} would turn into: registeredHooks.format_numbers( registeredHooks.format_dates(value) ).

Either way, it is definitely more complex behavior, harder to implement and harder to understand... and I still feel that it is less readable :-)

Justin Hileman

The new "helpers" feature of Mustache.php (2.0-dev) makes this super awesome:

Given a set of formatters:

<?php

class DateFormatters {
    const SHORT_DATE = 'd/m/Y';

    public function __isset($key) {
        return method_exists($this, '_'.$key);
    }

    public function __get($key) {
        return array($this, '_'.$key);
    }

    public function iso8601($date) {
        return $this->parse($date)->format(DATE_ISO8601);
    }

    public function atom($date) {
        return $this->parse($date)->format(DATE_ATOM);
    }

    public function short($date) {
        return $this->parse($date)->format(self::SHORT_DATE);
    }

    private function parse($date) {
        return new \DateTime($date);
    }
}

Register them as helpers:

$m = new Mustache;
$m->addHelper('date', new DateFormatters);

And use 'em in all your templates!

{{# date.short }}{{ item.createdAt }}{{/ date.short }}

(note that this is fully supported by the current spec: it doesn't require anything besides lambda support and dropping a bunch of helpers in the top-level context stack frame)

Anders Hellerup Madsen
ahem commented March 20, 2012

@bobthecow It doesn't fully follow the current spec, does it? Is the dotsyntax in the spec now? Also, dropping helpers into the top-level context stack frame isn't either, i believe, but that is sort of what I think most of us here is getting at anyways.

How does Mustache.php solve the issue of multiple formatters active at one time? Or how would the name be rendered in a template like this:

{{# date.short }}{{ item.Name }} created at {{item.createdAt}}{{/ date.short }}

Would item.Name be passed to the date formatter? Would the "created at" string?

Justin Hileman
  1. It's fully spec-compliant. dot notation is part of the spec as of v1.1.0 (March 4, 2011).

  2. There's room for the PHP implementation of helpers in the spec: how you get things into your rendering context is of no concern to the spec. It's very language and implementation specific. In Ruby, you could add helpers by monkeypatching your Mustache class to add the formatter properties.

  3. The whole thing would be passed to the formatter, just like any other lambda. Instead, you would do: {{ item.Name }} created at {{# date.short }}{{item.createdAt}}{{/ date.short }}

Anders Hellerup Madsen
ahem commented March 20, 2012

Ah, okay. This is different from what @groue have implemented in GRMustache then. If you can't put anything between the tags, then it doesn't make sense to have this behavior implemented with sections - and I still stand by that {{ item.createdAt|date.short }} is much easier to read.

Also rendering the value, and then parsing it again with the helpers is not exactly best case is it? I would expect that in some cases information would be lost before the helper function got a chance to format it.

Justin Hileman

I was just pointing out that it's possible to implement without extending the spec.

That said, the pipe style does look a lot cleaner. I could get on board with a "pipe notation" that is (mostly) syntactic sugar for chaining lambda callbacks, the same way "dot notation" is (mostly) syntactic sugar for chaining nested section tags.

Where these:

{{ foo.bar.baz }}
{{ foo | bar | baz }}

are both a shorthand for

{{# foo }}{{# bar }}{{ baz }}{{/ bar }}{{/ foo }}

but in the case of dot notation, it does the current context stack restrictions, and in the case of pipe notation, it passes the previous value to the next lambda.

This would save the to string and back trip, and would clean up templates, especially when running multiple filters.

Gwendal Roué
groue commented March 20, 2012

Ah, okay. This is different from what @groue have implemented in GRMustache then. If you can't put anything between the tags...

Indeed GRMustache's proposal can deal with {{#format}}...{{date}}..{{#foo}}...{{otherDate}}...{{/foo}}...{{/format}}.

And this can not be acheived with lambdas. Some other hooks had to be introduced.

While I was still stuck with the lambda frame, the most I could do was to implement something like @bobthecow, that is to say a lambda that render and use their immediate content.

@bobthecow, what about checking GRMustache API ? It's not funny to read a documentation, but at least there is a documentation: https://github.com/groue/GRMustache/blob/master/Guides/sample_code/number_formatting.md

It's fully spec-compliant.

It looks that your helpers render raw rendered strings (that is to say, strings that you trust will not be touched by the Mustache engine) : unfortunately, this is not a spec-compliant behavior.

What you may have missed, and no one could blame you, is that the spec requires lambdas to output a template string that will be parsed by Mustache engine, no the rendered string itself : https://github.com/mustache/spec/blob/master/specs/~lambdas.yml#L28

Justin Hileman

What you may have missed, and no one could blame you, is that the spec requires lambdas to output a template string that will be parsed by Mustache engine, no the rendered string itself.

Sorry, yeah, I realized that after I wrote the example :)

Mustache.php is spec-compliant, and it works just like you say. You would have to pre-render the string, so the body of parse would handle that in a full (not off-the-cuff and typed into a <textarea>) implementation :)

Justin Hileman

@groue I have seen the GRMustache API, and I have fundamental problems with it :)

I'd much prefer the syntax to use pragmas rather than sections when it's changing the meaning of {{ }} inline like that.

raw: {{float}}

{{%FORMAT percent}}
percent: {{float}}
{{%FORMAT default}}

{{%FORMAT decimal}}
decimal: {{float}}
{{%FORMAT default}}
Gwendal Roué
groue commented March 20, 2012

@bobthecow You're right. Actually, I really don't like that lambdas can not output anything that will be rendered raw. You always have to make sure your lambda won't output any {{, in case you would trigger unintentional Mustache formatting. This means that you can not output data that comes from untrusted source.

Gwendal Roué
groue commented March 20, 2012

I have seen the GRMustache API, and I have fundamental problems with it :)

I'd be happy to know them.

changing the meaning of {{ }}

???

Justin Hileman

???

It's saying "Between {{#THIS_TAG}} and {{/THIS_TAG}} treat all {{ of_these }} differently". That's inconsistent with the way sections work elsewhere in mustache. It feels more like a compiler directive to me, which is why I'd prefer pragma instead of section syntax.

Anders Hellerup Madsen
ahem commented March 20, 2012

The spec does specify that "Lambdas used for sections should receive the raw section string" (here), so I guess, if one were to implement a lambda that would treat 'variable blocks whose value is a number' different from the rest, one could just do that.

This isn't exactly how GRMustache implements it, but I believe the effect is similar. The formatterstacks could be considered helpers for building lambdas like this, I guess. @groue should of course correct me on this :-)

As I read the spec, however, the context object isn't supposed to be accessible from inside the lambda. The lambda is supposed to receive only the text inside it's own section unrendered, so it would be something like {{ number }}.

Without access to the context object, it isn't possible to format anything, and even with access to the context object, it still needs to implement it's own Mustache parser (or have some sort of deep knowledge of it's parent parser, so it can reach into it like GRMustache does).

IMO this is not a viable option - the goal should be that a user could provide a formatter like this (JavaScript):

{ decimal: function (x) { return typeof(x) === 'number' ? x.toFixed(2) : x; } }

and, with that attached somehow, expect it to format a template like this:

{{% decimal }}{{ pi }}{{/decimal}}

or this:

{{ pi|decimal }}

into the expected "3.14".

Sam Pullara

How about something that is simply rewriting the current syntax in a more terse format. For example, you might write:

{{#format_isodate}}{{myDate}}{{/format_iso}}

but allow it to be rewritten, with no change to the viewmodel, as:

{{#format_isodate myDate/}}

And have it literally do the same exact thing, just with a more compact syntax.

Gwendal Roué
groue commented March 21, 2012

Without access to the context object, it isn't possible to format anything, and even with access to the context object, it still needs to implement it's own Mustache parser (or have some sort of deep knowledge of it's parent parser, so it can reach into it like GRMustache does).

@ahem : I can not find the @pvande (spec writer) comment about this, but if I remember well, lambdas do not have access to context, but they can still render their "inner content" (have access to a function that renders their template string with the current context). @bobthecow lambdas are hence legit.

{ decimal: function (x) { return typeof(x) === 'number' ? x.toFixed(2) : x; } }
(1) {{% decimal }}{{ pi }}{{/decimal}}
(2) {{ pi|decimal }}
(3) {{ decimal pi }} (@spullara proposal)

(1): I can't see how the x parameter would get the pi value, since this syntax allows to write {{%decimal}}...{{pi}}...{{e}}...{{/decimal}}, without any ovbious way to fill the x parameter.

(2) and (3): Why not, and let's admit that Mustache 2.0 already exists, and is Handlebars (since (3) is Handlebars syntax).


GRMustache splits its hooks in two sets: the regular lambda hooks, that can alter the template string, and the "delegate" hooks, that can alter the values about to be rendered.

Mustache lambda hooks say: "I'm about to render this template string. Should I render something else?".

GRMustache value hooks say : "In case you'd be interested, I'm about to render (value) in response to the "name" key. Should I render something else?"

I want to make it clear to anybody that those value hooks are a work in progress, not a formal proposal. I just wanted that my library users (including myself) can render their damned data, without writing too much boilerplate code, while maintaining compatibility (that is to say, with a clear frontier between regular Mustache, and GRMustache add-ons).

Handlebars is not a perfect solution, neither. Especially, the way they render array indices looks handy as long as the implementation language allows to extend any object with any key (https://gist.github.com/1048968) : with javascript, they can add their indices to the array items before they are rendered, so that the {{index}} tag renders them. Unfortunately, more rigid languages like Objective-C do not allow this very easily. GRMustache can render array indices without altering the viewmodel objects like Handlebars: https://github.com/groue/GRMustache/blob/master/Guides/sample_code/counters.md

Anders Hellerup Madsen
ahem commented March 21, 2012

(1): I can't see how the x parameter would get the pi value, since this syntax allows to write {{%decimal}}...{{pi}}...{{e}}...{{/decimal}}, without any ovbious way to fill the x parameter.

@groue That was why I used % instead of # in the tag. The idea is that %-blocks should signify something like the GRMustache hooks you have implemented. As you say, they work different from regular mustache hooks, so I really do feel they should have different syntax.

My proposal for hooks like that would be that instead of the type-based protocol you have implemented (NumberFormatters work for number, DateFormatters for dates and so on) every variable tag would just pass it's value through all active helpers before printing it. The helper functions are then responsible for returning the original value if they don't intend to alter it.

(2) {{ pi|decimal }}
(3) {{ decimal pi }} (@spullara proposal)

I like these solutions for their simplicity. This is easy to understand, easy to implement and easy to use. They don't allow users to register global helpers like your implementation does, though, which may be a weakness.

Oh, and regarding the @bobthecow lambdas, they may be legal, but they still depend on parsing the default string representation of a value back into the original value before formatting, and that may not be possible in all cases. And it is is slightly weird behavior anyways, isn't it?

Gwendal Roué
groue commented March 21, 2012

@ahem Got it, thanks for the explanation and the feedback. Now I need to let these fresh ideas boil in my head :-)

Gwendal Roué
groue commented March 21, 2012

Oh, and regarding the @bobthecow lambdas, they may be legal, but they still depend on parsing the default string representation of a value back into the original value before formatting, and that may not be possible in all cases. And it is is slightly weird behavior anyways, isn't it?

@bebthecow's lambdas depends on PHP weak typing : "3.14" the string and 3.14 the number are the same. In other languages, we'd have to 1. render the inner content, 2. convert to number, and 3. format. That's pretty contrived, I agree with you.

I won't blame him, though. Mustache spec, as it is, forces us to hack around like that. I'm really happy we're having this discussion today, so that a clear light is shed on current limitations, and that today's hacks look like what they really are: hacks.

Pieter van de Bruggen
Owner

From a use-pattern perspective, the correct way to model this problem is by using a method (not a lambda), as @janl suggested.

class ViewModel < Mustache
  def initialize(price)
    @price = price
  end

  def msrp
    '$' + Math.round(@price, 2)
  end

  def global_msrp(currency)
    "#{Math.round(@price, 2)} #{currency}"
  end
end
  This thing costs {{msrp}} ({{#global_msrp}}USD{{/global_msrp}}).

Methods (again, not lambdas) have access to the context they were declared in, and so can usefully look up the data that's adjacent to them in the context stack. Methods can also receive the raw template string when used in a section. This lets you cleanly couple your formatting logic to the data you're presenting. This is the way Mustache expects you to handle this today -- your view model is deliberately more than just data, but also encompasses your presentation logic.

From an approachability perspective, the lack of helpers / formatters / data-driven functions is something that is often raised. I feel like I understand the desire, but remain unconvinced that it's the right decision for the language. I would happily change my mind if someone could show me a real-world(!) use case that Mustache cannot currently serve.

Sam Pullara
Anders Hellerup Madsen
ahem commented March 21, 2012

@pvande From a technical perspective, I don't feel comfortable with helpers/formatters that are directly attached to the models. I think everyone can see the usefulness of, say, a site-wide date or currency formatter, instead of having to implement these formatting methods on every viewmodel class (which would lead to duplicated and hard to maintain code).

I think all of this boils down to that separting concerns between template, viewmodel and whatever generates the viewmodel, is no simple matter. I feel that the viewmodel should really just be data, and that the template should be responsible for presenting this data. You clearly feel differently.

My thoughts on this is that both the viewmodel and the template should be completely free from code, and whatever logic is needed while rendering the template (loops, if/else, formatting and simple transformations of data) should either be a feature of the template language or kept, as simple as possible, in a single, shared, place.

From an approachability perspective, the lack of helpers / formatters / data-driven functions is something that is often raised. I feel like I understand the desire, but remain unconvinced that it's the right decision for the language. I would happily change my mind if someone could show me a real-world(!) use case that Mustache cannot currently serve.

I don't know what real world examples to show if, say, a way to format dates or currencies or whatever, shared between several templates and viewmodels, isn't enough to convince you.

It seems like the demand for a helper feature in Mustache is there, though - not just from me, but based on the fact that Handlebars exists, and that several Mustache implementations already have added this one way or another. Also, @janl may suggest using methods, but he also writes in his Mustache 2.0 document that "Handlebars implements helpers. They are a good idea and we should outright steal them :)".

To @spullara

Because of the way scopes work in mustache it is pretty simple to put the helpers in the base scope and not pollute the view models at a lll.

@spullara That depends on the language. I had something about inheritance and extensionmethods in the first verison of my comment, but I removed it because it isn't true. I apologize for messing up the order of post...

But anyway, a helper such as the one @pvande demonstrates in his comment cannot be shared in any way. It is hardcoded to only work for the "ViewModel" viewmodel, and it only works for the "msrp" property. If I would like to add another currency value to the viewmodel, I would need to add another, similar formatter for that field, and if I would like to use this formatter on some other viewmodel I would have to do it all again. I think that is duplicated and hard to maintain code.

I don't think there is any easy way to pass an argument to the formatting function, that could be used to select the property to format, except writing it in the text inside the section, and IMO that isn't very pretty, and again, it isn't easy or even possible to select properties based on string representations of propertynames in all languages.

Gwendal Roué
groue commented March 21, 2012

@pvande : 1- How can we relieve the burden of writing dozens of stub methods/properties when there are many values to format?

2- I don't get how the Mustache subclass can format nested values, such as person.pet.price, for instance, in a template such as {{#people}}{{#pets}}{{price}}{{/pets}}{{/people}}. I'm curious about how it should be written. I'd be happy seeing some actual code.

Last point: not all Mustache implementations use a "viewmodel". Some render actual raw models. This is especially the case when the host language makes it painful (read, not practical) to create a specific class for each rendering. This is also the case when we render a data hierarchy. Hence the prevalent topic, in this thread, of providing "pollution-less" ways to achieve some features. Hence, also, my question number 2.

David Santiago

I would say a real-world use case is the one that Shopify apparently had when they wrote Liquid: They wanted their users to be able to customize the appearance of their section of the site. There's no way you could give adequate control over the style without being able to make stylistic choices about the data inserted into the template, from the template. Most of the ideas being discussed here don't seem to include room for arguments to the helper functions, which would be a shame.

Pieter van de Bruggen
Owner

@ahem The role of the viewmodel is to house the data and logic being dispayed. This serves to enforce a separation between display logic (conditionals, iteration) and serialization / presentation logic. While other templating languages simply consume data, Mustache chooses to be deliberately different on this point.

This isn't for the sake of being contrary: your templates can be analyzed and optimized, your presentation logic is localized to the viewmodel (as opposed to across dozens of non-code files), and the viewmodel is an artifact you can test.

That's not to say that your viewmodel must have behavior. In the simplest cases, a hash more than suffices for your viewmodel, and most Mustache implementations handle this perfectly. In fact, the referential transparency of Mustache names means that in many cases, you could replace a "rich" viewmodel with a precomputed hash and notice no difference. (This is no longer true if your viewmodel mutates state, or if you use methods for sections.)

It's tempting to argue that creating rich behavior-full viewmodel objects, then, is a waste of time -- after all, if you can get the same results by simply building up a primitive data structure, what's the draw? The same argument applies to writing documentation or tests, but many of us have accepted that the time spent writing those things is valuable, even though it may double or triple the repository size without adding new functionality. Mustache promotes viewmodels over data objects for the same reasons: clarity, maintainability, and accountability.

As for the issue of keeping the transformations localized, most languages have mechanisms for doing just that. Static utility classes, reusable code modules, and first-class functions can all be used to effectively distribute behavior across a large number of objects. When viewmodels are code, reusability is a code-facing problem, with standard solutions.


Helpers are tricky. Their primary purpose is to push logic down into the template, which Mustache abhors. At the same time, it's the tool that's most familiar to most developers already, and it's frequently harder to transition from a primitive data object to a rich viewmodel than it is to argue for helper functions. Even if every Mustache user were writing expressive viewmodels, we'd likely still be having this discussion because of the strong temptation to push formatting behavior down the point of use. Mustache believes that viewmodels -- not templates -- are the point of use.


@groue To address your concerns:

1) Metaprogramming can go a long way towards that end.

2) Each person's pet should be a viewmodel capable of formatting the price appropriately.

3) If a Mustache implementation is capable of invoking methods, looking up properties, and fetching from hashes by key, then it uses a viewmodel. (If it isn't, it's not a Mustache implementation.) That's not to say that every Mustache implementation has a base class to inherit from, or even necessarily calls it out in the documentation. A viewmodel is simply an object graph that is capable of presenting itself, and that accurately describes the "data" argument consumed by every Mustache implementation.


@davidsantiago That's certainly an interesting case. I may need to think on that for a while...

Anders Hellerup Madsen
ahem commented March 21, 2012

@pvande This is an interesting dicussion. I feel that helpers are a necessary tool precisely because it would increase maintainability without adding logic to the template - two of the points you make against it.

your templates can be analyzed and optimized, your presentation logic is localized to the viewmodel (as opposed to across dozens of non-code files), and the viewmodel is an artifact you can test.

I think everyone will agree that putting actual codeblocks into templates (if that is what you mean by having untestable logic across dozens of non-code files) is a really bad idea. Fortunately no-one is suggesting that.

I don't feel that this:

var userViewmodel = {
    name: "Billy Bob",
    birthday: new Date('1981-12-02'),
    birthday_isoformat: function () { return isoformat(this.birthday);  },
    birthday_readableFormat: function () { return readableFormat(this.birthday); }
}
var postViewmodel = {
    title: "A great post",
    content: 'Lorem ipsum....",
    publicationDate: new Date('2012-03-21'),
    publicationDate_isoformat: function () { return isoformat(this.publicationDate); },
    publicationDate_readableFormat: function () { return readableFormat(this.publicationDate); }
}

var template = "{{# user }}{{ birthday_readableFormat }}{{ /user }}";

is any more maintainable or testable than this:

var userViewmodel = {
    name: "Billy Bob",
    birthday: new Date('1981-12-02'),
}
var postViewmodel = {
    title: "A great post",
    content: 'Lorem ipsum....",
    publicationDate: new Date('2012-03-21'),
}

var template = "{{# user }}{{ birthday|readableFormat }}{{ /user }}";

Note the two templates - my point here is that this does not add logic to the template, currently the template (or template writer, at least) still has to decide wether he wants to write bithday_readableFormat or birthday_isoformat. With helpers added the decision will become birthday|readableFormat or birthday|isoformat. In my opinion, no difference there - no added logic.

On the models however, the addition of helpers will remove the need to add the boilerplate dateformatting properties, and thus remove the need to test it. I see this as a good thing - the logic is in the readableFormat() and isoformat() functions, so they are the ones that should be tested.

Yes, in a dynamic language like JavaScript I could easily write some kind of adapterfunction like this:

function addDateformatters (obj, datename) {
    obj[datename + "_isoformat"] = function () { return isoformat(this[datename]); }
    obj[datename + "_readableFormat"] = function () { return readableFormat(this[datename]); }
}

addDateformater(postViewmodel, 'publicationDate');
addDateformater(userViewmodel, 'birthday');

but why should I have to? All of this is helpers implemented already! It's just boilerplate code that you have to write everytime you want to use it. I strongly feel that this is functionality the templatingsystem should provide, it should not be left as an exercise for the user to complete time and time again.

The other thing is, that this is really only that easy because JavaScript is an extremely dynamic language. In a language like C# or Java this might not be so easy, and then the boilerplate code will become a real problem. By adding a formalized hook to add this kind of helperfunctions, you would be able to simply the viewmodels and improve the maintainability of them - without adding logic to templates or sacrificing testability.

Gwendal Roué
groue commented March 22, 2012

Despite pvande's reluctance to acknowledge that not everybody writes trivial templates, and that users deserve consideration, I'd be happy working with all men of good will.

Justin Hileman

Since Real Artists Ship, here's where I'm leaning:

https://github.com/bobthecow/mustache.php/compare/feature/filters

The assumptions here are that:

  1. Pipe notation is analogous to dot notation — it can be thought of as syntactic sugar for nested sections.
  2. Filters are just variables in the normal context stack.
  3. Values are not coerced into strings (or escaped) until they come out the other end of the pipe.
  4. Unlike nested sections, the first value in the pipe is fetched prior to passing it to the next lambda.
  5. If the first value in the pipe is falsey, or if any other value is not a lambda, the pipe evaluates to an empty string.

Also:

  • Filters don't get parameters. That's logic in your template. See the commentary above regarding {{ foo | date.iso8601 }} vs {{ foo | date: "%j. %F %Y" }} (or whatever).
  • Since this is a non-spec feature, it is toggled with a {{%FILTERS}} pragma tag.

In code:

<?php

$mustache = new Mustache_Engine;

$tpl = <<<TPL
{{%FILTERS}}
{{ greeting | case.lower }}, {{ planet | case.upper | !! }}
TPL;

$mustache->render($tpl, [
    'greeting' => 'Hello',
    'planet' => 'world',
    'case' => [
        'lower' => function($value) { return strtolower((string) $value); },
        'upper' => function($value) { return strtoupper((string) $value); },
    ],
    '!!' => function($value) { return $value . '!!'; }
]);

// hello, WORLD!!

Or (more likely if you're using Mustache.php's helper injection feature):

<?php

$mustache = new Mustache_Engine;
$mustache->addHelper('case', [
    'lower' => function($value) { return strtolower((string) $value); },
    'upper' => function($value) { return strtoupper((string) $value); },
]);
$mustache->addHelper('!!', function($value) { return $value . '!!'; });

$tpl = <<<TPL
{{%FILTERS}}
{{ greeting | case.lower }}, {{ planet | case.upper | !! }}
TPL;

$mustache->render($tpl, [
    'greeting' => 'Hello',
    'planet'   => 'world',
]);
David Santiago

This is very close to what I was thinking as well. The one argument I would have is about not allowing arguments to the filters. Picking a date format is a great example of why you would want that feature. There's way more ways to format dates than you could reasonably want to write no-argument functions for. In fact you might write a small universe of these argument-free functions with names that were essentially replacing their arguments. If you want to call this logic in the template, I guess maybe so, but at least it's not application logic, reaching into a database or writing code to process the request in the template or anything like that. Not any more so than if you wrote a "filter" function that did those things and had no arguments. You could already write a rogue lambda function to do those things if you were so inclined (in fact, the spec on lambdas has a test that demonstrates how to write a lambda that actually updates application state: https://github.com/mustache/spec/blob/master/specs/~lambdas.yml#L56). Allowing filter arguments would make it easier to write compact code on the app side and easier to select the output you want on the template.

Pieter van de Bruggen
Owner
pvande commented July 27, 2012

@bobthecow That's not too shabby; here are a few thoughts.

  • Nested sections are always rendered outside-in, so this is not directly analogous. Then again, dotted names are not directly analogous to nested sections either (failed section lookups don't allow subsequent lookups to bubble).
  • I'm curious about why a falsey initial lookup would preclude subsequent filters.
  • What happens when the initial lookup is a lambda? Is the lambda poached for its value, or is the raw lambda passed through the filter chain? What happens when a filter returns a lambda - when it it called?
  • I'm pretty sure that supporting object literals as filter properties would be a sign of madness, but I'm less certain that allowing additional lookup names as parameters is a disservice. Parameters certainly support a less stateful ViewModel, which seems desirable to me. It's certainly possible to do this while being non-evaling; I'm not sure what the difference between non-evaling and logic-free is in practice, though. Thoughts?
  • It starts to get a bit more difficult to debug longer filter chains -- more keys to lookup means more chances to miss.
  • How do these filter chains work in Sections? Inverted Sections?
  • Nitpick: I'm assuming you mean that the first value in the pipe is fetched before being passed along, since it's not actually being interpolated into the template until after it's passed through the entire pipeline.

Thanks for investing the time in this. :)

Gwendal Roué
groue commented July 27, 2012

GRMustache has taken a similar route.

Actually, GRMustache has not extended the syntax of Mustache as much as {{planet | uppercase | !!}}. One still has to use sections in order to apply one's filters: {{#!!}}{{#uppercase}}{{planet}}{{/uppercase}}{{/!!}}.

However, despite this syntactic difference (which could be eluded with using | as some syntactic sugar), GRMustache filters provide with quite useful features as well on top of those seen above:

  • since they apply to a full section, they can filter as many inner tags as they want: {{#uppercase}}{{greeting}} {{planet}}!{{/uppercase}} -> "HELLO MARS!".
  • since they apply to full sections, they can be nested.
  • they are provided with more information, and particularly the key that has provided the value, and the position of the tag in the template. This is wonderful for debugging. {{person.pet.name}} fails? GRMustache hooks know if the pet has no name, or if the person has no pet, or if no person was found at all, and where exactly this tag can be found.
  • there is an implicit filter that applies to the full template rendering. This allows to extend Mustache features without any explicit markup.

In the documentation, you will find that all hooks are paired in methods named "willDoSomething", "didDoSomething" - this is required by the memory management in Objective-C (one has to be able to tear down what he has initially done). There are links to sample code as well.

Regarding a few @pvande questions :

I'm curious about why a falsey initial lookup would preclude subsequent filters.

Not the case in GRMustache filters

What happens when the initial lookup is a lambda?

1-arity lambdas are values just like others. But they have no meaning until executed, and filters can not execute them, lacking any useful context. So, basically, filters can not do nothing else but returning them untouched, or encapsulated in another lambda.

0-arity lambdas are evaluated before entering the filter.

It starts to get a bit more difficult to debug longer filter chains

Without any notion of "call stack", that's sure. It's the same difficulty as debugging deeply nested sections. I believe this subject could be left to nice implementors that would provide this information to the hooks.

How do these filter chains work in Sections? Inverted Sections?

Filters take values, and output values. Eventually the last value enters the Mustache rendering code, which processes it with classic rules.

Thanks for investing the time in this

You're welcome.

Pieter van de Bruggen
Owner
pvande commented July 27, 2012

@groue Taking a look at your implementation, it looks like your lambdas take a Section object instead of a string ... interesting. That definitely takes us into the realm where it's impossible for us to validate things in a language agnostic way, but it does (rather conveniently) allow you to sidestep many difficult questions (e.g. should the lambda receive the raw template string or the rendered one?). It also gives you additional control over the output, the context stack, potentially data lookup... While an extended syntax for this may also be desirable, I think this approach might just solve most of the problems we've been looking at here, in addition to a few others that have nagged at me for a while.

So here are my questions for you:

  • Is it important for implementations to have a consistent interface for such an object?
  • Is there a clean way to extend this to lambdas used for interpolation?
  • How might specifications for these interfaces be described in a language-agnostic way?
Justin Hileman
  • Nested sections are always rendered outside-in, so this is not directly analogous. Then again, dotted names are not directly analogous to nested sections either (failed section lookups don't allow subsequent lookups to bubble).

True. It even points that out in the documentation on dot notation... That's why I said it "can be thought of" rather than saying "is" :)

Just like dot notation is a more handy approximation of nested sections, this is a more handy approximation of nested lambdas. E.g.:

{{# foo }}{{# bar }}{{ baz }}{{/ bar }}{{/ foo }}

~=

{{ baz | bar | foo }}
  • I'm curious about why a falsey initial lookup would preclude subsequent filters.

It seemed to be consistent with the way falsey values are handled elsewhere. I guess it's inconsistent with the analogous nested sections though, as the {{# bar }} and {{# foo }} sections would indeed be passed an empty string — assuming the inverted inside-out resolution that we're glossing over for the sake of simplicity :)

That said, I'm not sold on short-circuiting the filter pipeline when it hits a falsey value. You'll see that I implemented it the other way first, but changed my mind when I started messing with date formatting... The naïve implementation in PHP would give you a formatted version of the unix epoch if you passed it a falsey value, which would then require the implementer to check for truthyness in every filter lambda. That said, I'm not sold either way, and I'd be more than happy to change mine around.

  • What happens when the initial lookup is a lambda? Is the lambda poached for its value, or is the raw lambda passed through the filter chain? What happens when a filter returns a lambda - when it it called?

The initial lookup is resolved to a value (but not coerced into a string if it's not already). At each subsequent filter, it's assumed that the lambda will return something which should be passed directly to the next filter. Due to my answer to the question above, the chain is short-circuited in the event that any filter returns a falsey value, but like I said, I'm not sold on this bit.

  • I'm pretty sure that supporting object literals as filter properties would be a sign of madness, but I'm less certain that allowing additional lookup names as parameters is a disservice. Parameters certainly support a less stateful ViewModel, which seems desirable to me. It's certainly possible to do this while being non-evaling; I'm not sure what the difference between non-evaling and logic-free is in practice, though. Thoughts?

My take on almost every argument I've heard for parameters: make a ViewModel. That's exactly where your statefulness and logic belongs. It's a companion class to go along with a template and encapsulate all the code that you're saving by removing it from your template. If you're into the whole idea of logic-free templates, really make 'em logic-free.

I'm willing to concede filters, because they're super-useful. But I think parameters are going too far.

  • It starts to get a bit more difficult to debug longer filter chains -- more keys to lookup means more chances to miss.

True. Caveat emptor.

  • How do these filter chains work in Sections? Inverted Sections?

Meaning how do they work in a tag like {{# foo | bar | baz }}?

I haven't implemented them there. In interpolation, I figured they're something of a special case for escaping. The value at the start of the chain is in some format, and eventually it needs to be a format fit for rendering as a string. This isn't required for sections (or inverted sections) since they don't need to format anything. They're just 0, 1 or many element lists.

  • Nitpick: I'm assuming you mean that the first value in the pipe is fetched before being passed along, since it's not actually being interpolated into the template until after it's passed through the entire pipeline.

Yes. I meant fetched. I was trying to post that on a really sketchy airport wifi connection and the form kept timing out. I'll retcon that into what I meant, not what I said :)

Justin Hileman bobthecow referenced this issue in bobthecow/mustache.php July 27, 2012
Merged

Filters implementation. #102

Gwendal Roué
groue commented July 27, 2012

@groue Taking a look at your implementation, it looks like your lambdas take a Section object instead of a string ... interesting. That definitely takes us into the realm where it's impossible for us to validate things in a language agnostic way, but it does (rather conveniently) allow you to sidestep many difficult questions (e.g. should the lambda receive the raw template string or the rendered one?). It also gives you additional control over the output, the context stack, potentially data lookup... While an extended syntax for this may also be desirable, I think this approach might just solve most of the problems we've been looking at here, in addition to a few others that have nagged at me for a while.

So here are my questions for you:

Is it important for implementations to have a consistent interface for such an object?

@pvande, my opinion on this take is no, for four reasons:

  • an implementation should be able to give the library user more information than required by the spec (debugging information, for instance).

  • an implementation should be able to provide required information via language features. For instance, if a language support closures, there is the possibility that some information does not need to be provided as a function argument.

  • an implementation should be able to implement underspecified behaviors, or optimistically specified behaviors. Mustache lambdas are in this category (how is the lambda supposed to be able to get the inner rendering of the section?).

  • an implementation should be able to work around specification bugs. Mustache lambdas are in this category (because of their relation to the "set delimiter" tags, and their inability to avoid processing {{tags}} that would come from program input). If GRMustache allows for the specification behavior, it must explicitely chosen by the user. Under this view, GRMustache lambdas have two facets, and require custom versatile parameters.

The implementation should only tell what information it requires the library user to be provided with. Complying implementation would provide this information, in the form they want, and maybe more.

I admit that this ruins the embedding of PHP/Ruby/etc in the specification tests. The embedding of C/C++/compiled/etc was compromised from the start, anyway.

Is there a clean way to extend this to lambdas used for interpolation?
How might specifications for these interfaces be described in a language-agnostic way?

Just like the best specifications out there: with well-written words illustrated by some sample code in a clear, concise, and widely-known language. YAML falls short on this topic.

A few last words. It's funny that you ask about GRMustache lambdas on a thread about filters. The two concepts should be clearly distinguished:

  • lambdas are the objects able to process the template fabric, able to add tags, modify the raw template text.
  • filters (delegates in GRMustache vocabulary) are the objects able to process the view model values.

Although they both are "callable", they do not need the same kind of information to perform.

For instance GRMustache lambdas take a "section" object that mainly provides with the section innner string, and the ability to render the section in the current context. Filters get a value, the key providing this value, and debugging information.

Gwendal Roué
groue commented July 27, 2012

@bobthecow,

As previously said, my filters can apply on a full section (being able to filter all tags in a section), and there is an implicit (without markup) filter that applies to the full template (able to filter all tags in a template and its partials).

This has a huge advantage: it solves for good a recurrent request from Mustache users: "advanced" loop rendering.

With GRMustache filters, you can render {{#items}}{{index}}: {{name}}{{^last}}, {{/last}}{{/items}} into "1: ham, 2: oranges, 3: bread" (noticed the missing last comma?), without preparing the view model for the index and last keys. Sample code

I wish this feature would be considered as a breakthrough in a parameter-less Mustache syntax, and a huge service provided to the library user.

Of course, should Mustache take Handlebars path, with {{#each_with_index items}} parameterized sections, I would have more difficulties defending my take.

Gwendal Roué
groue commented July 28, 2012

I must confess my mistakes, and I join @bobthecow's take, with apologies for my previous message.

  1. I was wrong believing that template-wide and section-wide filters where necessary: with the pipe syntax, one can apply a filter to as many tags as he wants, removing the need for global filters. One could also say that "explicit is better than implicit", and that behaviors triggered by some far enclosing filter section is not desirable.

  2. I was also wrong thinking the pipe syntax could be seen as a syntactic sugar on top of fundamentally section-based filters (believing {{value|filter}} is equivalent to {{#filter}}{{value}}{{/filter}}). Actually, this reasoning fails for two reasons:

    • the support for {{#filter}}...{{value1}}...{{value2}}...{{/filter}} would be mandatory, leading to filters compelled to apply to all inner tags. This has already been rejected in the first point.
    • it gets worse with filtered sections: {{#value1|filter}}{{#value2}}...{{/value2}}{{/value1|filter}} can't be equivalent to {{#filter}}{{#value1}}{{#value2}}...{{/value2}}{{/value1}}{{/filter}}, because the filter would need apply to all inner tags, leading to value2 being filtered as well, and this would ruin the implicit locality of the {{#value1|filter}} syntax (it looks like only value1 would be filtered).

(Filtered sections are my preferred feature, since they allow to implement "power" collections, my pet subject, with {{#items|with_index}}...{{index}}...{{/items|with_index}}.)

So: my lib will join the filter train described here, I will happily help solving corner-case questions, and I will nevertheless keep my template-wide and section-wide "delegates", because they provide with a few features users can find handy. As for today, I've identified debugging templates (locating unexpected values), and providing default values for missing keys.

Gwendal Roué
groue commented July 28, 2012

Voilà, GRMustache implements {{%FILTERS}}{{value|filter|...}}.

id uppercaseFilter = [GRMustacheFilter filterWithBlock:^id(id value) {
    return [[value description] uppercaseString];
}];
id data = @{
    @"name": @"Arthur",
    @"uppercase": uppercaseFilter
};

// Returns "ARTHUR"
[GRMustacheTemplate renderObject:data
                      fromString:"{{%FILTERS}}{{name|uppercase}}"
                           error:NULL];

Sections can also be filtered: {{#value|filter|...}}...{{/value|filter|...}} (yeah, "power" collections are supported: {{#items|with_index}}...{{index}}...{{/items|with_index}} can work.)

Here are a few issues or conclusions brought by this implementation:

TLDR:

  • The closing tag of filtered sections is very long, and people won't like that.
  • The filtering should not mess with the context stack.
  • We may need to look up for filters in a separate global table, isolated from the view model.

Long version:

  1. The closing tag for filtered section can be quite long (see above). Regarding the trend in some mustache implementations to support shorcuts like {{\}} to close a section, I don't expect much applause if we keep on requiring the identity of opening and closing expressions.

  2. The context stack is left untouched by the filtering process.

    Precisely: the filtering is part of an "expression evaluation". This evaluation yields a value to section and variable tags, which render and/or mess with the context stack. I like this strict separation of concerns. First evaluate, second render that value.

    Here are a few other arguments against having the filtering mess with the context stack. Rendering {{value|filter}}, we would have two options: either the value and the filter enter the context stack in that order; either filter enters first, then the value.

    • if the filter enters first, it could "override" the value. This would be quite a surprise, especially because we read from left to right, and that it looks that value should have precedence, just as is {{#first}}{{second}}....
    • if the value enters first, if could "override" the filter. This can be seen as a feature, but also as an awful mess, because the developper could never, looking at a template, trust the consistency of the filters he reads.
    • eventually, after the filtered value has been computed, both the initial value and filter would leave the context stack, and let the filtered value replace them. So anyway both initial value and filter are temporary objects have no real purpose entering the stack.
  3. We have two problems by having filters searched just like regular values in the context stack.

    • First, a performance problem: rendering {{name|uppercase}} when we are at the 20th level of section nesting can slow down a lot the extraction of uppercase, which is much likely defined at the bottom of the stack.
    • Second, we again have the problem of trust led by the fact that any object that enters the context, thinking no harm, could define its own uppercase, leading to bugs that would be quite tricky to fix.

    I don't really know how to escape this issue, but by providing filters "on the side", at a global level separated from the view model. Handlebars does this already. It looks that @bobthecow does as well.

Pieter van de Bruggen
Owner
pvande commented July 28, 2012
Gwendal Roué
groue commented July 30, 2012

@pvande @bobthecow I've updated the filter syntax in GRMustache. Here is the documentation: https://github.com/groue/GRMustache/blob/filter_chain/Guides/filters.md

TLDR:

  1. Syntax is now {{ filter filtered }} instead of {{ filtered | filter }},
  2. Avoiding the PHP syndrome
  3. Filters are provided in a separate and constant context stack
  4. There is a standard filter library
  5. Some problems remain.

Long version

  1. I was looking at the Liquid standard library. They implement a first filter which returns the first element of a collection. Applied to Mustache with the pipe syntax, this would give {{items|first}}.

    I feel very very incomfortable with {{items|first}}, because it looks so close to {{items.first}}, that itself looks like a method call. And one would wonder why this first stuff is a filter in the first place, when it just looks like it should be implemented as a method of the receiver:

    `{{ items.first }}`     # looks natural
    `{{ items|first }}`     # shouldn't I write `items.first` instead?
    

    Since filters really are functions, I think they should look like so. We should get rid of the mental confusion described above. I tried the colon, in order to avoid heavy parentheses:

    `{{ first:items }}`
    

    Unfortunately, for expressions mixing colons and dots, this is not very readable:

    `{{ first:basket.items }}`
    `{{ math.abs:coordinate.x }}`
    

    Finally I was satisfied with the white space:

    `{{ first basket.items }}`.
    `{{ math.abs coordinate.x }}`.
    

    Plus it clearly shows the precedence of space vs dot: first basket.items is clearly first(basket.items), not first(basket).items. first:basket.items is much less clear

  2. We all hate languages that have inconsistant notion of what is an expression. You wants to look like PHP?

    Remember that {{foo.bar}} is not the same as {{#foo}}{{bar}}{{/foo}} because the first one performs a "scoped" search?

    So, what is the "scoped" equivalent of {{# filter foo }}{{bar}}{{/ filter foo }}?

    {{ (filter foo).bar }}? Suggestions welcome.

  3. Isolating filters from the view model solves those performance and context stack issues, and it also leaves room for the filter standard library:

  4. The Liquid filters can take arguments. GRMustache filters do not. Hence there are less predefined filters.

    The interesting ones are blank? and empty?. empty? is a solution to the frequent request from being able to test for a collection before rendering it (see #14, #22, #23): {{^ empty? items }}<ul>{{# items }}....

Gwendal Roué
groue commented July 30, 2012

@pvande

We had that before, and a couple dozen Mustache implementations were written against it. Unfortunately, about the most they have in common is that they all use doubled curly braces to represent interpolation. We also had the Ruby implementation as a "gold standard", and several implementations used that as their implementation guideline -- but the Ruby version hasn't yet hit 1.0, and hasn't maintained the same semantics since those implementations started. I drafted the spec in the hopes of creating a set of language-agnostic unit tests to aid implementors, on the assumption that executable examples would be less ambiguous and more approachable than English descriptions of the rules. I'm happy to be challenged on that assumption -- maintaining a robust language-agnostic shared test suite is hard -- but I've found the lack of ambiguity to be invaluable in every implementation I've done.

That's pretty interesting. I was enclined to think that you were reluctant to write a formal grammar, or to use the MUST, SHOULD, etc words, and that this spec hardly deserved its name.

OK. I appreciate the fact that you are experimenting.

Anyway, I don't see how the lambdas and filter specifications will avoid plain English.

Justin Hileman

@groue There is a lot of prior art for using pipes for "filters": Unix, Linux, Liquid, Twig, Django/Jinja, and many, many more. We're not breaking new ground by having {{ foo | bar }} mean "take foo, pass it to bar". It also means chained filters make sense: {{ foo | bar | baz }} makes lots of sense, but what does {{ baz bar foo }} mean without parentheses? Is that {{ baz(bar(foo)) }} or {{ baz(bar)(foo) }}? We don't need to invent any new paradigms here, we can simply adopt the ones that work everywhere else.

Another complication of using whitespace delimiters for filters is that whitespace is valid in variable names in the Mustache spec, as well as in most Mustache implementations.

Justin Hileman

For what it's worth, liquid's first pipe doesn't fit with mustache. It's stepping across the line into "logic in the template".

I really don't see any reason to support filters on sections. Filters, in my opinion, are a concession for allowing a small amount of output formatting logic in templates. Filters on sections (or inverted sections) are not used for output formatting, they're used for adding logic to templates. This is very un-mustache, and probably unnecessary.

Sam Pullara

I find that most people propose filters when they don't realize that you aren't supposed to use your data objects directly behind the mustache template. Explaining that you have view code associated with each template and that is where filtering/formatting/etc takes place usually clears it up with the users I have worked with.

Gwendal Roué
groue commented July 30, 2012

@bobthecow @spullara

Regardless of the syntax (pipe vs space, ordering), filtered sections are quite useful when one wants to process collections. For instance, for adding support for rendering 0-based indexes, 1-based indexes, for conditionally rendering a section for the first or the last item in a loop, etc. The usual response of Mustache "lawyers" is: prepare your view model. The usual response of Mustache users is to open issues in this repository and in the Implementation repos. I mean, seriously. This topic MUST be addressed in some way. Pragmatism is necessary. I will repeat it as many times as it needs: no way Mustache should provide baked-in support for these "power collection" features. But would it provide hooks for the library user so that he can render his loops the way he wants, with his code, the world would be wonderful. Section filters allow that.

About the "naturality" of pipe syntax : where @bobthecow and I can have a disagreement, it's on the fundamental nature of what's in a tag. For me, tags contain expressions. For him, they contain statements. The fundamental difference between them is that expressions can and should be composed, when statements can arbitrarily be final. Hence my question about the "scoped" filtered expression, that you missed, @bobthecow. See above, I'm sure you'll agree on its relevance.

Pieter van de Bruggen
Owner
pvande commented July 30, 2012

@bobthecow @groue @spullara I'd like us to take a step back for a few minutes and think about what we actually care about when we say "logic in the template". I've created a wiki page to facilitate this discussion (please post to the bottom, and sign your updates). After we've hashed some of this out, we'll be in a better place to actually document what it is we care about and what we don't. I'd also like to call on @defunkt for his contributions to that discussion.

Pieter van de Bruggen
Owner
pvande commented July 30, 2012

@bobthecow I can't speak to whether filters for Sections are necessary, but they are probably relevant for consistency. The simplest useful case I can come up with is an empty? filter for lists; in languages without a coherent way to determine that emptiness up-front, it's difficult to write a template like this:

{{# errors | empty? }}
  Please resolve these errors:
  {{# errors }}
  * {{ message }}
  {{/ errors}}
{{/ errors | empty? }}

At the same time, the "official line" is that errors_is_empty? should be exposed by your ViewModel. And indeed, in some ways, it is: the emptiness of errors is an intrinsic property of that list, which can be queried in Ruby (errors.empty?) and less directly in Javascript (errors.length), but not everywhere.

In fact, the worst case scenario I can come up with for filtered sections is taking a single scalar value and passing it through a filter chain that merges that dynamically builds up a collection from that data value, which might feel a bit like data generation, except that all the data is still coming from the ViewModel. More pragmatically, there's the with_index filter to expose iteration numbers (for emulating a join), with lambdas you could build a proper join filter (which practically begs for parameterization), and each_pair, which could transform a hash into an unordered list of key/value pairs. These are all things that would be commonly used if provided, and there may be many further abuses of this feature.

I've avoided incorporating filters thus far because I wasn't sure whether they could be used to violate the Logic-Free constraint. I'm becoming more convinced that they generally might not. Without having a concrete definition for Logic-Free, however, it becomes difficult to measure.

Pieter van de Bruggen
Owner
pvande commented July 30, 2012

@spullara I absolutely agree; most requests for filters come from not wanting use Mustache "properly". Having said that, there is certainly room for them to exist in a Non-Evaling world; there may also be a way for them to exist in a Logic-Free world without compromising the intent of a testable ViewModel.

Pieter van de Bruggen
Owner
pvande commented July 30, 2012

@groue While I'll agree that composable expressions are more powerful and expressive, I would need to see a real-world use case for them before I was convinced that they were necessary. Also, what @bobthecow has been driving at is not a "statement" driven model, but a non-composable expression model -- each (possibly dotted) name still resolves to something, which may be a value, a lambda, or a method; each | sends the lhs named value to the rhs named code and returns a new value. In character, there's no difference between foo | bar | baz and baz(bar(foo)).

Justin Hileman

I can't speak to whether filters for Sections are necessary, but they are probably relevant for consistency.

Not necessarily. There's no need for {{{# foo }}} because whether the interpolated value of a variable is escaped is only relevant when actually interpolating the variable. I feel like filters are a more general case of the same thing. Given data from the ViewModel, I would like to use an alternate display format.

Another issue: other than empty? I haven't been able to come up with a single use case for filters on sections that didn't amount to variable assignment or data generation. Keep those out of your logic-free templates :)

Pieter van de Bruggen
Owner
pvande commented July 30, 2012

@bobthecow While I generally agree (please, contribute your perspective to the wiki page!), I think join ends up being a clear example of something that, while solvable with a content and template change, is less than ideal, and requires neither as a filter.

# without a filter
list:
  - name: Abe, last?: false
  - name: Ben, last?: false
  - name: Cal, last?: false
  - name: Dan, last?: false
  - name: Eve, last?: true
{{! without a filter }}
{{# list }}{{name}}{{^ last? }}, {{/ last? }}{{/ list }}

{{! or worse (if sections could receive the rendered text) }}
{{#remove_trailing_comma}}{{# list }}{{name}}, {{/ list }}{{/remove_trailing_comma}}
# with a filter
list:
  - name: Abe
  - name: Ben
  - name: Cal
  - name: Dan
  - name: Eve
{{! with a filter }}
{{# list | join }}{{name}}{{/ list }}

There's no added data, in this example, just output modification by convention (joined by ', ').

I'm actually having a difficult time coming up with potential filters to do variable assignment or arbitrary content generation. At best it seems like you can get content generation according to pre-defined rules, which may or may not be substantially different.

Justin Hileman
# with a filter
names: [Abe, Ben, Cal, Dan, Eve]
{{! with a filter }}
{{ names | join }}
Justin Hileman

please, contribute your perspective to the wiki page!

I'm on my cell phone. When I get a chance to sit down at a computer I'll add stuff to the wiki :P

Pieter van de Bruggen
Owner
pvande commented July 30, 2012
Justin Hileman
{{# foo | bar }}{{ baz }}{{/ foo }}

If foo is a list, hash or object, which of these happens?

  • foo is fetched, and passed to bar. The return value of bar is treated as the section context, and the section is rendered as usually.
  • foo is fetched, and the section is rendered as usually. The rendered section is passed to bar. The return value is output.

If foo is a lambda, which of these happens?

  • The foo lambda is passed to bar. The return value is used as the section context and the section is rendered as usually.
  • The foo lambda is evaluated. The return value is passed to bar. The return value is used as the section context and the section is rendered as usually.
  • The section template is passed to foo lambda, as higher order sections are. The return value is passed to bar. The return value of bar is rendered as a mustache template and output.
  • The section template is passed to foo lambda, as higher order sections are. The return value is rendered as a mustache template. The rendered output is passed to bar. The return value of bar is output.

What happens when foo is falsey? Does the section get rendered? How 'bout if falsey foo is passed to bar? If bar returns something truthey, does the section get rendered?

What happens when foo is a scalar (thus used for boolean sections) but bar returns a list?

Pieter van de Bruggen
Owner
pvande commented July 30, 2012

@bobthecow Ah, now my own questions get turned on me. :)

Alright then, here's a proposal:

  • Names may include filters, separated by |.
    • The | "operator" has a lower precedence than the . "operator".
      • e.g. {{ foo.bar | baz.wux }} would pass foo.bar to baz.wux.
  • {{# foo | bar }} behaves as {{# bar(foo) }} (not {{ bar(#foo) }})
    • foo is fetched and passed to bar.
      • If bar is a scalar, or is not found, the result is falsey.
    • Only the value at the end of the filter chain is considered by the tag.
      • For Section tags
        • If bar(foo) is listy, the section is rendered against each element of the list.
        • If bar(foo) is a lambda, the section behaves as a lambda section.
        • If bar(foo) is truthy, the section is rendered against it.
        • If bar(foo) is falsey, the section is not rendered.
Justin Hileman

@pvande That seems consistent with how I was thinking about filters for interpolation. It also seems internally consistent with the rest of Mustache.

Given that interpretation, the "name" of the tag would be foo.bar | baz.qux. I like the idea of a short closing name, e.g.

{{# foo.bar | baz.qux }}stuffs{{/ foo.bar }}

... but that does make it a bit tougher to build a naive parser. Thoughts?

Pieter van de Bruggen
Owner
pvande commented July 30, 2012

@bobthecow The name of the tag would indeed be foo.bar | baz.qux.

Regarding simplified close tags, I don't know how often I've seen Rubyists write code like:

module Foo
  class Bar
    def baz(list)
      list.each do |x|
        # ...
      end # each
    end # def
  end # class
end # module

While I may find it a little distasteful, I can't deny the merit. On the other hand, the comment itself is freeform, and is usually only a subset of the original declaration. We should start another thread for the discussion of simplified close tags, but it seems fairly reasonable to me. (This should also play into the notion of "else" tags a bit.)

I'll also throw out there that I'm not aware of a naive parser that passes the spec currently...

Justin Hileman

One more quirk. Given:

{{^ foo | bar }}FAIL{{/ foo }}

What happens when bar is not found?

I'm beginning to think that falsey, scalar and missing filters are an exceptional case and should be treated as a runtime exception, similar to what you'd do if a higher-order section returned invalid mustache syntax.

Pieter van de Bruggen
Owner
pvande commented July 30, 2012

I'll buy that.

Justin Hileman

I'll also throw out there that I'm not aware of a naive parser that passes the spec currently...

Mustache.php v1.x was really close :P

Pieter van de Bruggen
Owner
pvande commented July 30, 2012

@bobthecow I'll bet you a nickel it was the {{a}}{{a}}...{{/a}}{{/a}} test case that tripped you up. :)

Justin Hileman

@pvande Nope. It was whitespace edge cases. Those were the only spec fails.

Pieter van de Bruggen
Owner
pvande commented July 30, 2012

@bobthecow Ah, well... Good on you then! (I've seen a number of Regex-based "parsers" bite it on that case.)

Justin Hileman

That said, I don't regret switching to a real parser. The new hotness is at least 20x faster — orders of magnitude faster with template caching enabled.

Gwendal Roué
groue commented July 31, 2012

Hi @pvande, @bobthecow, @defunkt. I wrote something about filters, that I wish you all read: https://github.com/groue/GRMustache/blob/filter_chain/WhyMustacheFilters.md.

Gwendal Roué
groue commented July 31, 2012

@pvande, @spullara, you'll find in pvande's wiki page a pragmatic interpretation of logiclessness that may move your positions on what's "forbidden" or not "realized".

Pieter van de Bruggen
Owner
pvande commented July 31, 2012

So, here's a new angle.

Filters necessarily mean that we cannot prohibit this: {{ sql | query }} or {{ ruby | eval }}

Lambdas already give us some subset of this: {{#query}}DELETE FROM table;{{/query}} or {{#eval}}FileUtils.rm_rf '/'{{/eval}}

This seems (to me) to be directly counter to the Logic-Free principle, in addition to the "user-safe" notion. The latter might be considered an acceptable risk if you're not allowing user-written templates, but it's certainly unintentional behavior. But at least it's sandboxed to the data provided to the context stack, right?

data = {}

# { } -> Hash -> Object -> [Kernel] -> Kernel -> Kernel.eval
key = "class.superclass.included_modules.last.eval"

template = "{{# #{key} }}puts 'Hello'{{/ #{key} }}"
Mustache.render(template, data)

This only fails in today's implementation because of a fluke -- methods that don't have an arity of exactly 1 aren't passed arguments. Kernel.eval has an arity of -1 (it claims to take zero or more arguments in 1.8.7; actually, it has one or more optional arguments). If (more likely, when) the Ruby implementation sends content strings to methods with optional arguments ... game over.

Now, what can we do about this:

  • We can refuse to dispatch to methods that weren't declared on the object type itself. (Hash.new() no longer responds to class, sandboxing is maintained)
  • We can rescind the ability to execute code from templates. (Unfriendly, but there's no safer option.)
  • We can rescind our claim of being "user-safe" and our claim to enforcing Logic-Free templates.
  • We can reduce "user-safety" and "Logic-Free" to lifestyle choices; not something inherent to us, but something we choose to do.

I am not happy with these options.

Pieter van de Bruggen
Owner
pvande commented July 31, 2012

@pvande, @spullara, you'll find in pvande's wiki page a pragmatic interpretation of logiclessness that may move your positions on what's "forbidden" or not "realized".

Your position has been noted.

Gwendal Roué
groue commented July 31, 2012

Lifestyle, for Christ's sake, or you'll never be able to look at yourself in a mirror, having contributed to such an evil library. Do you really want to ship a VM with Mustache, that would safely run the nasty code of users?

Gwendal Roué

GRMustache has updated its syntax for filters, so that it's consistent with what I think are good filters :

  • composable
  • compatible with "scoped" lookup.

As @pvande noticed, the best syntax known so far, which has those two properties, is f(x).y. Check out the doc!

Gwendal Roué

The GRMustache fitler documentation describes:

  • what happens for missing filters or objects that are not callable (boom: a runtime error)
  • "filter namespaces", for users who like {{ math.abs(x) }}
Gwendal Roué

@pvande it would be fair of you to acknowledge my propositions, or write a rebutal. This silence doesn't fit your position of maintainer of this repo. Could @defunkt share his thoughts with you on the subject?

Gwendal Roué

Anyway. I've shipped for good, updated my filters proposal with many elements from other Mustache implementations, and since some implementors may not feel at ease with parsing ’f(x).y’ expressions, provided a grammar and a state machine that's easy to implement.

Hendy Irawan

+1 for this!

Gwendal Roué

GRMustache has just shipped with support for "meta filters": filters that return filters.

Release notes: https://github.com/groue/GRMustache/blob/master/RELEASE_NOTES.md#v542

Sample code: https://gist.github.com/3803707

Gwendal Roué

GRMustache has just shipped with support for "variadic filters": filters that take several arguments.

Release notes: https://github.com/groue/GRMustache/blob/master/RELEASE_NOTES.md#v55

Sample code: https://gist.github.com/3803707

Gwendal Roué groue referenced this issue in twitter/hogan.js October 30, 2012
Closed

Piped Helpers #113

Gwendal Roué groue referenced this issue in janl/mustache.js November 02, 2012
Open

Added piped helpers #261

Gwendal Roué groue referenced this issue in janl/mustache.js November 02, 2012
Open

Implement Helpers. #139

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.