is there a way to force assetgraph-builder to process script tags as html templates?

[mreinstein]

I'm using backbonejs in one of my apps, and have several script tags that contain html templates. Here's an example of one of the template scripts, embedded in my index.html:

<script type="text/template" id="about-template">
        <h1>welcome back, <%- username %>!
        <img src="img/icn-logo.png" class="whim-logo" alt="logo" />
</script>

in my backbone view, I basically load this as you might expect:

data = $('#about-template').html()
template = _.template data
$('#about').html @template( { username: 'mike' } )

Is there some way I can use the GETSTATICURL to cause these references to be satisfied?

I'm sorry to keep flooding you folks with github issues. : (

[Munter]

If you set the content type of the script to be text/html Assetgraph will treat it as a html fragment and populate the relations from it, so you will also have icn-logo.png in the graph.

And you certainly shouldn't be sorry about posting issues. Most of this just highlights cases we haven't spotted before or lack of documentation. It all helps us improve :)

[papandreou]

It seems like a lot of people are using <script type='text/template'> for this. Added support for it here: assetgraph/assetgraph@33c3bea

This will be part of the next assetgraph-builder release.

It still remains to be checked whether those <% ... %> things survive the HTML parser, stay tuned...

[mreinstein]

Yeah, it's the defacto content type that most people use in the backbonejs examples I've seen. Most browsers don't know how to deal with type="text/template" so it's generally left untouched. text/html is probably ok for html fragments but for templated content I think it's the way to go.

[papandreou]

Hmm, unfortunately, they don't:

console.log(require('jsdom').jsdom('<h1>welcome back, <%- username %>!\n<img src="img/icn-logo.png" class="whim-logo" alt="logo" />').outerHTML);

Output:

<h1>welcome back, </h1>

I assume those <% ... %> things can occur everywhere, not just where a tag or an HTML comment would be allowed?

[mreinstein]

the <% ... %> and <%- ... %> and <%= ... %> can occur pretty much anywhere in the block

[mreinstein]

the other thing to keep in mind is there are as many/more javascript templating engines than there are projects that use them. : o

[papandreou]

Damn. That'll make it hard to support. The code that finds relations from an HTML document/fragment relies on being able to parse the document with an HTML parser, work on the DOM, and then reserialize it later. This works great with Knockout.js and Angular.js templates, but template languages such as this one are basically sugar for concatenating strings, so it can't really be parsed in a structured way.

If we can find a way to transform those <% ... %> fragments into something an HTML parser will accept and still be able to put them back when reserializing the document, it'd be possible. Turning them into HTML comments would work in most, but not all cases:

console.log(jsdom.jsdom('<h1 class="<!--%="theClass--> otherClass">').outerHTML);

Output:

<h1 class="class">\n  <!--%="theClass-->\n otherClass"</h1>

Alternatively we could implement it as a separate asset type that recognizes relations on a best effort basis by regexp'ing on the non-dynamic fragments. Would that be useful?

[mreinstein]

I wouldn't get too stressed about supporting every type of template language. Unless significant numbers of people really start complaining about it not working for them, don't spend a lot of time on this. for example, when I opened this ticket, changing my script tags from text/template to text/html as @Munter suggested fixed the problem perfectly for me.

As a short term thing, I'd do 2 actions:

1. make text/template get handled the same way as text/html (just eval the script block the same way)
2. documentation on this behavior :)

those are both pretty easy to do, and they solve most problems. IMO assetgraph-builder is awesome even with all of the quirks i've run into, and you guys should focus on fixing the bigger problems first. : ) this isnt one of them

[papandreou]

changing my script tags from text/template to text/html as @Munter suggested fixed the problem perfectly for me.

The point is that your templates will break if you do that. At first, your image will be found and made part of the build, but when its url is changed to /static/<md5HexPrefix>.png, the contents of the template will be reserialized, and the <% ... %> fragments will be lost.

[mreinstein]

well I guess I don't understand about the implementation details but that is surprising. I would suspect that if you parse one of those templates blocks as html, and get a DOM back...if you identify any images or resources to replace, they would be around the template directives. I expect that in the dom they'll end up as text nodes. but maybe not.

If this can't work, then maybe another alternative is to use an html5 data attribute to hint at what template engine is used, and support the most popular few. I think it's a scenario where a handful comprise the bulk, and then there's a long tail. Underscore, jade, handlebars, moustache, and whatever ember uses are probably the big ones...

e.g.,

<script type="text/template" id="about-template" data-template-engine="underscorejs">
        <h1>welcome back, <%- username %>!
        <img src="img/icn-logo.png" class="whim-logo" alt="logo" />
</script>

[mreinstein]

another problem you'll have is you wont be able to eval those templates. They are frequently data driven, given stuff at runtime. So your best bet is to preserve the contents of templated sections, and not bother with trying to evaluate them. That is super, super messy...

[Munter]

We have to parse the dom to find any relations to assets.

@papandreou I think we should meet up and bounce some ideas off each other for handling templates. It seems to me that this could be the single biggest pain point that users will have. I was going to suggest a beer tonight anyway, so now you get the invitation on github instead, like a real geek.

[papandreou]

@Munter: I agree, we need to come up with some kind of strategy, and even if it turns out to be too messy to support those outgoing relations, it needs to be documented.

I'm out of town right now and will be for a while, family stuff. Let's have that beer when I'm back.

[mreinstein]

where are you guys located? I'm in the bay area, CA. If you're nearby would definitely like to meet up for a beer sometime.

[mreinstein]

ah, Denmark it looks like. :)

[papandreou]

well I guess I don't understand about the implementation details but that is surprising. I would suspect that if you parse one of those templates blocks as html, and get a DOM back...if you identify any images or resources to replace, they would be around the template directives. I expect that in the dom they'll end up as text nodes. but maybe not.

I was hoping that as well, but it seems like the HTML parser used by jsdom just discards them. Chrome's parser turns them into text nodes with the less and greater thans entitified. I'm pretty sure we can get 95% of the way by applying a transformation before parsing and after serializing.

ah, Denmark it looks like. :)

Yeah, unfortunately that does limit the attendance of our brainstorming/beer sessions :/

[mreinstein]

I think there is probably no way to do this completely reliably; template hinting seems like a necessary thing here. You have to somehow know what templating system someone is using before being able to parse it. and I think this is a completely reasonable thing; the current practice of just having text/template blocks lying around and then implicitly coding to a certain template engine is kind of crappy. I'd be fine with being able to specify per template block what it is data-template-engine="handlebars", or maybe alternatively have a commandline option like --html-template-engine handlebars

I wonder if there is already a proposed pseudo standard for specifying the type of template engine being used? I'd hate to re-invent the wheel on this.

[mreinstein]

this could be useful, provides lower level info on the text parsed into tokens:

https://github.com/tautologistics/node-htmlparser

[papandreou]

this could be useful, provides lower level info on the text parsed into tokens: https://github.com/tautologistics/node-htmlparser

That's the one jsdom is already using. They're stuck on version 1.x, though, and 2.0.0 was a complete rewrite. However, it seems like jsdom is considering a switch to a different parser, so it might resolve itself soon: jsdom/jsdom#482

[mreinstein]

Hey guys, any update?

[papandreou]

@mreinstein: Unless we come up with a tolerable way of getting most relations in this type of template right, I'm afraid this will be a "Won't fix". It's simply impossible to model the relations and cover all cases where the template won't survive a round trip through the HTML parser, eg.:

<div><%= "</div>" %> <span>foo</span>
<img <%= "src='img/icn-logo.png'" %> alt="logo">

... etc. So the question is whether we can come up with a solution that works well enough out of the box. It's not being worked on right now, though.

In the mean time I should probably revert the commit that treats <script type="text/template">...</script> as HTML if people use that type for this kind of template.

Annoying workaround (assuming that global variables are in scope when template fragments are evaluated):

<script>var whimLogoUrl = GETSTATICURL("img/icn-logo.png");</script>
<script type="text/template" id="about-template">
        <h1>welcome back, <%- username %>!
        <img src="<%= whimLogoUrl %>" class="whim-logo" alt="logo" />
</script>

If that doesn't work, let me know, and I'll try to think of something else :)

[papandreou]

In the mean time I should probably revert the commit that treats <script type="text/template">...</script> as HTML if people use that type for this kind of template.

Reverted in assetgraph/assetgraph@fa2e70d

[thbaja]

Hey Guys,

I came across this thread as I was having similar issues with underscore templates.

As @papandreou suggested, one could you use something like html comments for certain template use.

I chose to go with a Moustache like style, as suggested in the underscore documentation. It is also documented how you change the delimiters: http://underscorejs.org/#template

Maybe this can help you too, @mreinstein
Example:

_.templateSettings = {
  interpolate : /\{\{(.+?)\}\}/g
};

[Munter]

Running into this with TodoMCV's Marionette implementation as well.

Just adding this for reference. Sadly I have no new input on the matter.

These are the templates that get mangled:

<script type="text/html" id="template-footer">
    <span id="todo-count">
        <strong><%= activeCount %></strong> <%= activeCountLabel() %>
    </span>
    <ul id="filters">
        <li>
            <a href="#">All</a>
        </li>
        <li>
            <a href="#active">Active</a>
        </li>
        <li>
            <a href="#completed">Completed</a>
        </li>
    </ul>
    <button id="clear-completed" <% if (!completedCount) { %>class="hidden"<% } %>>
        Clear completed (<%= completedCount %>)
    </button>
</script>
<script type="text/html" id="template-header">
    <h1>todos</h1>
    <input id="new-todo" placeholder="What needs to be done?" autofocus>
</script>
<script type="text/html" id="template-todoItemView">
    <div class="view">
        <input class="toggle" type="checkbox" <% if (completed) { %>checked<% } %>>
        <label><%- title %></label>
        <button class="destroy"></button>
    </div>
    <input class="edit" value="<%- title %>">
</script>
<script type="text/html" id="template-todoListCompositeView">
    <input id="toggle-all" type="checkbox">
    <label for="toggle-all">Mark all as complete</label>
    <ul id="todo-list"></ul>
</script>

[mreinstein]

my solution is to work on a better templating language that is valid html. check it out! https://github.com/mreinstein/lure

[Munawwar]

And here is mine: https://github.com/Munawwar/Htmlizer/ (inspired by KnockoutJS).

[Munter]

Well, there certainly are a lot of options now :)