Skip to content

Y.Template and Y.Template.Micro - Generic template API and simple ERB/Underscore-style JavaScript templates #230

Closed
wants to merge 10 commits into from
@rgrove
rgrove commented Aug 25, 2012

While working on a new widget recently, I found myself wanting a string-based templating solution that was more advanced than Y.Lang.sub() or Y.substitute(), but without the overhead of Y.Handlebars. I needed support for interpolation, if/else branching, and looping, but only for a few very small templates. Underscore-style templates (more familiar as ERB-style templates to Rubyists) seemed like exactly the right fit.

These commits add a new Y.Template class which provides a generic template engine API, and a Y.Template.Micro static class which provides a simple micro-templating engine. Y.Template can be used to compile, precompile, render, and revive precompiled templates using Handlebars or Y.Template.Micro.

Y.Template

Using with Y.Template.Micro (the default template engine):

YUI().use('template', function (Y) {
    var micro = new Y.Template(),
        html  = micro.render('<%= data.message %>', {message: 'hello!'});

    // ...
});

Using with Handlebars:

YUI().use('template-base', 'handlebars', function (Y) {
    var handlebars = new Y.Template(Y.Handlebars),
        html       = handlebars.render('{{message}}', {message: 'hello!'});

    // ...
});

See the API docs for further details.

Y.Template.Micro

Provides simple ERB/Underscore-style micro-templating. Can be used standalone or via Y.Template.

Within a template, <% ... %> is used to insert a block of JavaScript code, <%= ... %> evaluates and prints an expression as HTML-escaped output, and <%== ... %> evaluates and prints an expression as raw, unescaped output. Properties on the data object passed to a template function are made available within the template on the data variable.

Here's a simple template that renders a list:

<ul class="<%= data.classNames.list %>">
    <% Y.Array.each(data.items, function (item) { %>
        <li><%= item %></li>
    <% }); %>
</ul>

And here's the code to compile the template and render it (assume listTemplate is a string containing the template above):

YUI().use('template-micro', function (Y) {
    var compiled = Y.Template.Micro.compile(listTemplate),
        html;

    // Render the compiled template to HTML.
    html = compiled({
        classNames: {list: 'yui3-list'},
        items     : ['one', 'two', 'three', 'four']
    });

    // And again, with a different set of data.
    html = compiled({
        classNames: {list: 'yui3-list'},
        items     : ['a', 'b', 'c', 'd']
    });
});

Precompilation is supported too, so Micro can be used to precompile templates at build time or on the server.

// `precompile()` returns a string containing JavaScript code that will evaluate
// to a template function. It can be written to a file, served to a remote
// client, etc.
var source = Y.Template.Micro.precompile(listTemplate);

Performance

Y.Template.Micro is significantly faster than both Underscore and Handlebars at both compiling and rendering templates. See http://jsperf.com/y-template-vs-others/6

Running Locally

The built template, handlebars, and yui modules are intentionally not included in this pull request in order to keep the diffs clean. Before testing this change, you'll need to build them.

$ cd src/handlebars && shifter
$ cd src/template && shifter
$ cd src/yui && shifter
rgrove added some commits Aug 24, 2012
@rgrove
rgrove commented Aug 25, 2012

Oh travisbot. When will you learn to read?

@davglass
YUI Library member

Ha! yogi picked up your tests automatically since they are in there ;)

I'll have to check and see if the new Travis Env var is there for pull requests. Then I can have shifter auto build the modules before running tests. I'll look into this tomorrow ;)

@ericf
YUI Library member
ericf commented Aug 25, 2012

I worry that this muddies the waters with respect to YUI and templates. Also, putting JavaScript in the templates might encourage people to do bad things; someone like yourself will do so wisely, but I worry about the average developer uses this as a hammer.

I think we should be encouraging people to move templates outside of their JavaScript code and this seems like it might encourage people to keep them internal (as does Y.Lang.sub().)

I'm curious to hear your arguments of why this should be in core vs. Gallery though — it might just be that this is bringing back horrible memories of ASP for me :)

@rgrove
rgrove commented Aug 25, 2012

@ericf: Long story short? Because today was the second time in the past few weeks that I've found myself writing a widget that needed templates more complex than Y.Lang.sub() could handle, but not as complex as Y.Handlebars would have been. After spending some time going down the path of precompiling Handlebars templates at build time and baking them into my modules (and then seeing how much my code size ballooned as a result), I decided we needed an option in between.

For me, Y.template() is the Goldilocks option. It's simple, small, fast, and powerful. That also means that, yes, you can use it to shoot your foot off. But YUI is already a pretty big gun, and this is just one of many bullets. If we wanted to protect everyone from themselves, we'd be politicians, not JavaScript developers. ;)

Y.Lang.sub() is great for strings. Y.Handlebars() is great for web pages and complicated views. I see Y.template() being great for widgets that don't need complex views, but do need to render HTML (like the list above) with simple conditional or looping logic.

Given a choice between doing something unholy with Y.Lang.sub(), doing something way overkill with Y.Handlebars, or doing something quick, simple, and readable with Y.template(), I'll choose Y.template() every time.

Why should it be in core? Well, beyond what I said above, if it's not in core, then the widgets I'm working on can't go into core either. And I think you might want them. ;)

@ericf
YUI Library member
ericf commented Aug 25, 2012

If precompiled Handlebars templates were easy to work with and the plumbing was baked into core (and Shifter), would you still feel the need for Y.template()?

If you assume the handlebars-base module was always going to be loaded (so don't count it), would the precompiled Handlebars template functions still add too much bloat to your Widgets?

@rgrove
rgrove commented Aug 25, 2012

@ericf: Even if handlebars-base (~3KB) were always loaded (which I find unlikely), a precompiled Handlebars template for the list example above would still come to 1,380 bytes, whereas the precompiled Y.template() template is only 260 bytes.

https://gist.github.com/3459956 vs. https://gist.github.com/3459993

Even for a tiny template, Handlebars has a lot of boilerplate overhead. Y.template() doesn't.

@ericf
YUI Library member
ericf commented Aug 25, 2012

@rgrove Gzipped the templates are 183B and 462B respectively. So in this case it's 60% smaller than Handlebars, as the template grows in size, the Handlebars boilerplate will hopefully become less of the overall size of a template.

The tentative plan I've began talking about with people is moving all Widget templates to Handlebars, precompiling the template(s) for a module into a separate YUI module as part of the build process, and making the template-module a dependency of the Widget.

I would like to see us be able take a single approach for HTML templates and I think the fact that you've created Y.template() means there's more to consider for smaller templates and how we should approach templates for all Widgets. It would be great to have a single solution we can use for everything, including for people to use for their app templates :-/

I think this means we should have this discussion now instead of in Q4 when this work on library-wide templates is scheduled.

@rgrove
rgrove commented Aug 25, 2012

@ericf: Actually, I'd expect the opposite to be true. Handlebars has significant boilerplate code for each {{token}}, whereas Y.template() doesn't. So as the size of the template grows, the Handlebars boilerplate will continue to balloon. Luckily, it's very repetitive so it should gzip well, but even so, that's significant.

I don't think there's anything wrong with having multiple templating solutions to meet different needs, as long as the reasons for using each are clear. Handlebars has its place, as does Y.template(). I see Handlebars being a more attractive solution for view management (partials and helpers are big wins there) and for server-side templates. But I think Y.template() is more attractive for the kinds of client-side widgets YUI has right now.

@lsmith
lsmith commented Aug 25, 2012

I expect Handlebars would be overkill for widgets in general, and including that much overhead for the (albeit unrealistic) "putting a single X on the page requires YK of YUI code?" story is bad for adoption. It reenforces the perennial "YUI is heavy/over-engineered" complaint.

Y.template feels like the right balance for use by widgets for markup creation. Adding vastly more logic in the templates (via Handlebars) is a direction we don't want to go because that easily balloons into PHP, and will likely hinder configurability of individual widgets. For simple string substitution, Y.Lang.sub is fine, but the lack of a compile step is a missed opportunity for widget performance. The security consciousness of <%= ... %> is a safe default for markup generation that saves boilerplate code and/or prevents developer forgetfulness. The conditionals I'm on the fence about, because they are the first step on the slippery slope toward Handlebars-esque feature richness and complexity. But I do think it's a reasonable place to draw the line in the sand and will prove practical and useful for widgets.

DataTable might benefit from Y.template.

@hojberg
hojberg commented Aug 25, 2012

Side discussion.
Should this be called something other than Y.template() ? I feel like Y.template should be used for something that is template-engine agnostic, and could be configured to use specific engines. Could make it a target for gallery modules that added new template engines.

@rgrove
rgrove commented Aug 25, 2012

@hojberg: I doubt we're likely to see (or need) something like that in YUI, but I'm open to other names if there's something that's clearly better.

@jshirley

I find myself agreeing with nearly all of the comments here. Y.template would discourage usage of Handlebars, when in many cases Handlebars would be the right solution. I can see in cases of widgets, the currently named Y.template.

However, having the name as global as Y.template without substituting the engine seems overly empowering to the (very slightly) more dangerous option.

I do think the name is key. Y.Handlebars exists specifically for this, but perhaps Y.Template as a top level namespace is better, and this can be something like:

var myTemplate = Y.Template.simple("Hello <%= name =%>!");
myTemplate({ name : 'World' });

(Then a wrapper for Y.Template.handlebars that calls Y.Handlebars.compile, for a consistent interface).

@davglass
YUI Library member
@juandopazo
YUI Library member

I understand the motivations, but I agree with Eric and Dav. It reminds me of my old PHP days as well. I don't like that it doesn't look at all like the existing options in YUI. So I ask myself, what do we need that's between Y.Lang.sub and Handlebars? Iteration? Mapping objects?

Why not Mustache instead? It's a subset of Handlebars and weighs much less:

  • Handlebars minified with Uglify: 29.5 KB
  • Mustache minified with Uglify: 4.39 KB
@hojberg
hojberg commented Aug 25, 2012

@jshirley exactly what I was thinking!

@rgrove
rgrove commented Aug 25, 2012

@jshirley: I'd be okay with moving this to the Y.Template namespace, and I think your comments tie in nicely with what Dav said...

@davglass: Y.Lang.sub() is still useful. Y.substitute() is useless, crappy, and needs to die. I'm not interested in adding a compat shim so that Y.template() can work like Y.substitute(), because I think the way Y.substitute() works is stupid.

I definitely see your point about Y.template() and Y.Handlebars needing to share common ground in how templates are compiled and loaded. I'll give this some thought. Whatever happens though, I don't want Y.template() to become something more complex than it is now -- there's huge value in a tiny (<1KB) module that can meet all simple templating needs with no confusing abstractions.

@juandopazo: What does Mustache provide that Y.template() doesn't to make it worth its 4.39KB? That's over four times the size of Y.template(), and even larger than the handlebars-base module needed to render precompiled Handlebars templates. If it's the syntax you like, then Y.template() can easily support that. Mustache's many shortcomings are why we ended up choosing Handlebars as YUI's heavy-weight templating solution. One of its biggest shortcomings is that templates are always interpreted at render time and can't be compiled, which would make Mustache both slower and bigger than using precompiled Handlebars templates.

I have to accept some of the blame for the current situation. Back when we were trying to decide on a templating solution for YUI, I had the urge to investigate Underscore-style templating, but ended up being swayed by the power and strict separation of concerns that Handlebars provided. In hindsight, I think Underscore-style templating would have been a better fit for YUI's client-side templating needs, and on the server where Handlebars templates make more sense it's easy enough to use the non-YUI Handlebars module.

I won't say I regret bringing Handlebars into YUI (or writing all that documentation -- gawd), but I do think it's not the best template language for client-side widgets, and I think Y.template() is.

@rgrove rgrove Generic Y.Template API and Y.template() -> Y.Template.Micro
Moves Y.template() to a static Y.Template.Micro namespace and
adds Y.Template, a generic template engine API that supports
Y.Handlebars, Y.Template.Micro, and any other template engine
that adheres to a simple API interface.
d3ce3d7
@rgrove
rgrove commented Aug 26, 2012

Okay dudes. My latest commit incorporates your feedback and adds a generic template API for both micro-templates and Handlebars. It also moves Y.template() to a static Y.Template.Micro namespace. I've updated the pull request description above with details of the new changes. Let me know what you think.

@okuryu
YUI Library member
okuryu commented Aug 27, 2012

I feel that need to add docs/* if pull in this module. And, I would like to expect to get benefit from this one, but I think to need to some notes about "JavaScript in the templates" on docs as Eric remarks.

@rgrove
rgrove commented Aug 27, 2012

I'll definitely add a user guide if this gets pulled in. Wasn't going to write one on spec though -- too much work if things change.

@hojberg
hojberg commented Aug 27, 2012

@rgrove great stuff!

@rgrove
rgrove commented Aug 28, 2012

Updated benchmarks: http://jsperf.com/y-template-vs-others/4/

Turns out Y.Template.Micro using the variable option to disable the use of with inside compiled templates is faster than both Underscore and Handlebars.

@davglass
YUI Library member

@rgrove I like the changes, great work as always!

@okuryu
YUI Library member
okuryu commented Aug 29, 2012

I'd like to share talk about this Pull Request.
Part 1: http://www.youtube.com/watch?v=dLZ3AGD5d1w
Part 2: http://www.youtube.com/watch?v=qO2xCmnY53g

@rgrove
rgrove commented Aug 29, 2012

I filed an enhancement ticket for shifter describing how I think it should support automatic template precompilation for YUI modules using Y.Template. Feel free to weigh in on that discussion as well: yui/shifter#11

@evocateur evocateur and 1 other commented on an outdated diff Aug 29, 2012
src/template/js/template-micro.js
+ // Replace the token placeholders with code.
+ .replace(/\ufffe(\d+)\uffff/g, function (match, index) {
+ return blocks[parseInt(index, 10)];
+ }) +
+
+ "';";
+
+ if (!options.variable) {
+ source = "with(data){\n" + source + "}";
+ }
+
+ source = "var $t='';\n" + source + "\nreturn $t;";
+
+ // Compile the template source into an executable function.
+ template = this.revive(new Function('Y', options.variable || 'data',
+ 'htmlEscaper', source));
@evocateur
evocateur added a note Aug 29, 2012

Continuing thought from davglass/shifter#11:

This is where Y.Template.Micro#precompile() falls down, in my view, when used as a precompilation engine for shifter. It would be preferable to pass { "source": true } or something as the options for compile(), and only revive() the generated function when source is false. (Conversely, when the source option is true, return only the generated source, as with Handlebars#precompile())

if (options.source) {
    template = "function(Y," + (options.variable || 'data') +
        ",htmlEscaper){\n" + source + "\n}";
} else {
    template = this.revive(new Function ('Y', options.variable || 'data',
            'htmlEscaper', source));
}
@rgrove
rgrove added a note Aug 29, 2012

This seems reasonable, although I don't think it has any direct bearing on what we were discussing re. shifter (except that this will make precompilation slightly more efficient). The question of whether precompilation should be possible without requiring YUI at all is still a separate issue, and I still think the answer to that is no.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
rgrove added some commits Aug 29, 2012
@rgrove rgrove Don't rely on `with`, don't do extra work during precompilation.
Removing the `with` block in compiled templates makes rendering way
faster, bumping Micro ahead of both Underscore and Handlebars in
rendering speed.
173bf22
@rgrove rgrove Make Y.Template.Handlebars an alias for Y.Handlebars 7d9b214
@rgrove
rgrove commented Sep 7, 2012

No movement on this pull request despite the two hour meeting we had last week in which the team consensus was that this should be merged.

What's blocking this?

@hojberg
hojberg commented Sep 8, 2012

I'd love so much to see this merged!

@nhusher
nhusher commented Sep 10, 2012

Read through the code this morning. I would like to see this in core, especially with the ability to plug multiple render engines into Y.Template.

@yzapuchlak

Personally, I don't see myself using the Micro templating language in the near future (handlebars is working well for my current needs), but I think the Y.Template engine framework is a very valuable addition as it would standardize how people go about creating modules (gallery or private) to add YUI support for any of the other popular (or niche) templating languages out there.

So I agree with many on this thread. This would be great to see pulled in.

@ericf
YUI Library member
ericf commented Sep 10, 2012

I will merge this in after our 3.7.0 code freeze (which we're in right now). @rgrove sorry that the window passed us by for getting this into 3.7.0, and I'd like to stick firmly to our code freeze process.

This will also give me a chance to play around with the new APIs and really push on them. Having Y.Template will work out nicely with this ticket #2532428, which is something I'll be tackling for the next development sprint.

Once the code freeze is lifted, I'll merge this in and we'll start planning the first 3.8.0 preview release. Getting a preview release out quickly will get Y.Template and Y.Template.Micro on the CDN, and more importantly, be easy for other developers to try out and give feedback.

@rgrove
rgrove commented Sep 10, 2012

Thanks @ericf!

@dmitris
dmitris commented Sep 19, 2012

is the freeze over now as 3.7.0 has been released?

@ericf
YUI Library member
ericf commented Sep 19, 2012

@dmitris We are still in a code freeze post release to make sure we don't have to issue any emergency patches.

@hojberg
hojberg commented Oct 2, 2012

Possible to get this merged?

@ericf
YUI Library member
ericf commented Oct 2, 2012
@hojberg
hojberg commented Oct 2, 2012

thanks @ericf !

@ericf ericf was assigned Oct 9, 2012
@ericf
YUI Library member
ericf commented Oct 15, 2012

I think that "template" is the wrong noun for the object returned from calling the Y.Template() constructor function. I found this confusing as to what to name my var which holds the instance object returned from Y.Template() while filling out the rest of the unit tests.

Really what we're doing is creating a template engine instance. The following seems more correct to me:

var engine = Y.Template.createEngine(Y.Handlebars),
    html   = engine.render('{{message}}', {message: 'hello!'});
@rgrove
rgrove commented Oct 15, 2012

@ericf: I think of it like this: Y.Template.Micro is an engine. An instance of Y.Template.Micro is a template.

Incidentally, what unit tests are you filling out?

@ericf
YUI Library member
ericf commented Oct 15, 2012

@rgrove Agreed, so what is the return value from Y.Template()? To me it returns a generic engine wrapper, therefore I see it as more of an engine factory than returning a template instance.

My updates to the unit tests are very simple, edge case tests which get template to 100% line, function, statement, and branch coverage! \o/

@rgrove
rgrove commented Oct 15, 2012

@ericf: Awesome, thanks for filling out the test coverage!

The return value from new Y.Template() is still a template, it's just a template that has some internal logic that tells it, at instantiation, what engine it should use. Y.Template is a generic engine wrapper, but an instance of Y.Template is a wrapped template.

@ericf
YUI Library member
ericf commented Oct 15, 2012

I'd argue that "{{message}}" is the template from the code snippet in my above comment.

@rgrove
rgrove commented Oct 15, 2012

@ericf: Agreed. "{{message}}" is a template, and Y.Handlebars.compile("{{message}}") is a compiled template. Y.Handlebars and Y.Template.Micro are engines, and the return value of new Y.Template() is an engine. I've made some clarifications in the doc comments.

@ericf
YUI Library member
ericf commented Oct 17, 2012

@rgrove I'll make you a deal… if you stub out a TOC for a Template/Template.Micro user guide, I'll fill in the details. I think we can get away with one user guide to service both Y.Template() and Y.Template.Micro. After I fill in the details you can copy-edit and add polish, if you wish (and have time to do so.)

@rgrove
rgrove commented Oct 17, 2012

@ericf You're too kind! I'll have something for you later today.

@rgrove
rgrove commented Oct 17, 2012

@ericf I pushed a first pass at a user guide TOC. Let me know what you think.

@ericf
YUI Library member
ericf commented Nov 6, 2012

This has been merged into 3.x.

@ericf ericf closed this Nov 6, 2012
@hojberg
hojberg commented Nov 6, 2012

Awesome! :dancers:

@rgrove rgrove referenced this pull request in yui/shifter Dec 19, 2012
Closed

Support template precompilation using Y.Template #11

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.