JS: Add support for EJS and Mustache-style templating dialects

[asgerf]

Adds support template files in the following dialects: EJS, Mustache, Handlebars, Nunjucks, Hogan, and Swig.

I've tried to keep the commits self-contained, but it's a large PR so it would help to have a high-level overview before diving in.

New sinks recognized

For an EJS snippet like,

<h1><%- title %></h1>

we now recognize title as an XSS sink, since <%- ... %> performs raw interpolation in EJS. For example, it would be vulnerable if instantiated like:

res.render('mytemplate', { title: req.query.title })

Even when a "safe" interpolation tag is used (one that escapes the output), there are a few cases where we recognize it as a code injection sink:

• In a code attribute, HTML entities are expanded prior to JS parsing, so the escaping is not effective, so we treat these as sinks.
• In a script tag, HTML entities are not expanded, which usually prevents breaking out of string literals, but if not safely wrapped in a string literal, we treat it as a sink.
• In the context of an AngularJS template, AngularJS will interpret the output, so we treat these as sinks as well.

Extracting placeholder tags

To support placeholders with non-trivial expressions, like <%- foo.bar %>, we extract the contents of such placeholders as JS with the same dialect we use for Angular2 expressions. (This dialect has internally been renamed to "angular-like template").

The regular JS parser has also been updated to recognize placeholder tags occuring in plain JS context, so we can parse script tags that contain such placeholders. For example, we now get flow from userdata to the innerHTML assignment here:

let data = <%= userdata %>;
elm.innerHTML = data.html;

Wiring up template instantiations and template files

To see what file is rendered by res.render('foo/mytemplate', { ... }), we have to match up the foo/mytemplate string with a file.

Here we use a heuristic algorithm which:

• Finds all files with the appropriate path suffix, such as foo/mytemplate.html, foo/mytemplate.ejs, or foo/mytemplate/index.html.
• From these candidates, we pick the one with the lowest common ancestor in the file hierarchy.

For example, given these files,

• src/lib/app.js containing a call res.render('foo/mytemplate')
• src/lib/views/foo/mytemplate.html
• src/lib/mytemplate.html
• src/docs/foo/mytemplate.html

The src/lib/mytemplate.html file fails the initial suffix test (foo/mytemplate).
The remaining template files have the following lowest common ancestor with the referencing file (src/lib/app.js):

• src/lib/views/mytemplate.html -> src/lib/
• src/docs/mytemplate.html -> src/

Since the first one has the lowest common ancestor (or equivalently, the longest common prefix) that one is picked as the target.

Evaluations

• Default slugs shows good performance.
• On 100 EJS-specific projects we find 11 code injection and 23 xss results (mostly TPs) and a bunch of other alerts.

[erik-krogh]

Wow, that's a mouthful.

Here is a partial review from trying to go through commit-by-commit.

[erik-krogh]

I can see that this class is needed, because the tests fail if I remove it.

But I'm not sure where it's needed.
I can't get it to fit with the API::Node usage in getTemplateInput, but maybe that's me missing something.
Could you elaborate in which class/predicate this API::EntryPoint is needed?

[asgerf]

It's needed for EjsIncludeCallInTemplate. By ensuring that the callee has an API node, the arguments to call have API nodes as well, which is needed in getTemplateParamsNode, which is needed in getTemplateInput.

Using EjsIncludeCallInTemplate directly in the definition of the entry point leads to negative recursion, unfortunately.

[erik-krogh]

Great, could you add a reference to EjsIncludeCallInTemplate in the qldoc?

[erik-krogh]

👍

Impressive work!

[asgerf]

Thanks for powering through another mega PR @erik-krogh! ✨