Templating language tags in javascript strings break syntax highlighting

[dotandimet]

When trying to implement syntax highlighting for Mojolicious templates (highlight.js pull request, mojo pull request), we found that highlight.js can't handle template tags inside javascript strings.

This problem also shows up with the ERB syntax (which I used as reference):

It doesn't happen with PHP tags, which are handled internally in the xml.js language definition:

I suspect this can only be fixed in xml.js, not by modifying an external language definition.

[isagalaev]

Good catch, thanks! I'll look into it as time allows but it might prove to be a real tough one…

[zicai]

same problem,when i use mustache tag {{ and }}

in markdown:
https://github.com/zicai/zicai.github.com/blob/master/_posts/2015-03-08-mustache-intro.md

with highlight.js
http://zicai.github.io/lessons/2015/03/08/mustache-intro/

the words in {{ and }} are missing

[isagalaev]

Removing 8.8 as it is indeed proving to be a tough one.

P.S. @zicai: the issue you mentioned has absolutely nothing to do with the original one. Moreover, it's not a scripting issue at all as those missing words are already missing from your HTML source.

[dajocarter]

@zicai If you need the mustache tags {{ }} to render, try wrapping them in {% raw %} {% endraw %}.
http://stackoverflow.com/questions/24102498/escaping-double-curly-braces-inside-a-markdown-code-block-in-jekyll

[codestitch]

@zicai - it didn't work. it only displays {% raw %} {{data.id}} {% endraw %}

[swatilk]

The issue with escaping two curly braces {{ }} still exists. Anyone know of a solution?

[lobo-tuerto]

I just stumbled into this one too.
In my case I'm working with a Vue.js app, the fence code that is throwing the error is:

```html
<template lang="pug">
v-app
  v-toolbar
    v-toolbar-title {{ $static.metaData.siteName }}
    v-spacer
    v-toolbar-items
      v-btn(flat exact :to="{ name: 'home' }") Home
      v-btn(flat :to="{ name: 'about' }") About

  v-container
    slot
</template>


<static-query>
query {
  metaData {
    siteName
  }
}
</static-query>
```

The error I'm seeing when enabling hljs inside markdownit is:

 ERROR  [Vue warn]: Error in render: "TypeError: Cannot read property 'metaData' of undefined"

When hljs is disable then no errors are thrown.

[lobo-tuerto]

It's as if it was trying to parse and execute the code inside {{ }}.

[lobo-tuerto]

This is related to Vue.

Solved by changing the hljs invocation from this:

hljs.highlight(lang, str).value

To this:

let highlighted = hljs.highlight(lang, str).value
highlighted = highlighted.replace(/{{(.*)}}/g, '{{ "\\{\\{$1\\}\\}" }}')
return highlighted

[marcoscaceres]

There I an alternative Vue language definition that might help (I've not tried it): https://github.com/highlightjs/highlightjs-vue

[joshgoebel]

I think this problem is misstated (or at the very least oversimplified):

Templating language tags in javascript strings break syntax highlighting

The reason this doesn't work is because template tags are NOT Javascript. :-) I'm not sure you should actually expect this to work IMHO. I'm not sure it's so great we added automatic PHP support to xml, though I guess that ship has sailed. :-) There is also a cost here. Now you can't build a working Hightlight.js with "xml" without bundling "php" as well, although perhaps the parser just quietly ignores missing languages.

But if that's the case I'm not sure how it can ever properly detect the END of the templating, since it can't know about any escaping being using inside it (ie):

<h1>
I'm just a <%= "HAPPY %>" bunch of more Ruby code here that isn't html %>
</h1>

Although I guess if you weren't actually using that templating engine then it wouldn't so much matter if it silently failed and also never matched.

---

The real problem is many (all?) template engines are generic. You can use ERB with JS, HTML, XML, JSON, CoffeeScript, and much more... so if we're going to add to one, we need to add to all - or we need to understand clearly why JS is such an exception compared to the others. AFAIK, ERB is more commonly used with HTML than JS - so if JS is an exception I think HTML would be a larger exception.

IE, this is NOT just an issue with the Javascript parser, and shouldn't be though of as such. Again, unless we first define why it's such an exception vs the other cases. We made need a better way to think about templating engines in general.

Perhaps it'd be best to flag a snippet as "php/html".... or "erb/js"...

[joshgoebel]

Since most major template engines "trump everything" (ie, the "start" macros can show up literally anywhere) perhaps there is some built-time option we could add to say "compile highlight.js with ERB support"... and then magically you can just toss ERB in the middle of any language - and it will "just work"...

If you look closely you can see even say the built-in support for PHP starts to fail if you look at edge cases, like PHP inside attribute strings... which should definitely be possible... you should be able to break in anywhere:

You can see PHP isn't detected inside the attribute string... and also here is another example of where this type of "one thing in another" goes south... the Javascript is detected as xml (because it includes PHP I suppose) so you don't get the JS syntax highlighting... And I'm willing to bet if you had gotten the Javascript highlighting then we wouldn't have gotten the PHP highlighting. :-)

Since <script> right now can fall back on 5 different parsers?

subLanguage: ['actionscript', 'javascript', 'handlebars', 'xml', 'vbscript']

VBScript, lol. :-) Does anyone still use that on the web?

---

I say all of that just to say: Lets make sure we really understand the actual core problem with templates here before we just start swinging our axes trying to solve it any which way. :-)

[joshgoebel]

I forgot (well, actually I never know) that in our own repo we have example of how you'd actually do this correctly, such as VBScript-HTML:

function(hljs) {
  return {
    subLanguage: 'xml',
    contains: [
      {
        begin: '<%', end: '%>',
        subLanguage: 'vbscript'
      }
    ]
  };
}

[joshgoebel]

@dotandimet You still around? I think what you want is your own language that's defined similar to the VBScript/HTML language. It's a bad idea (and simply not practical) to add every possible templating language under the sun to xml. Adding PHP to xml in the past was probably short-sighted without considering what that would look like long-term.

Also it becomes almost impossible when as soon as have different templating languages that share similar syntaxes... (it looks like ERB and Mojolicious are very similar). Having a separate language per templating engine removes all the ambiguity.

[joshgoebel]

Closing as asked and answered and lack of activity.