New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
html/template: add support for template strings (backticks) #9200
Comments
I really should have seen that coming since I wrote the spec strawman: https://code.google.com/p/js-quasis-libraries-and-repl/ This doesn't actually introduce a new execution vector for client-side code since the backticks are significant inside a block of JavaScript code, not inside HTML. Automatic interpretation of '{{' inside <template> elements does that. (sigh) Will fix. |
Might also be an issue inside of JS attributes, as in this demo: https://play.golang.org/p/iahSK9oqLc - pops up alert in modern browsers |
#29406 was about malformed template output since backticks aren't parsed properly. I've re-titled this issue to be about the overall support of this ES6 feature, and closed the newer issue as a duplicate. I'm marking it as NeedsDecision, just in case anyone disagrees with adding this feature. It's part of the JS standard, so as long as |
I'm not sure if anyone else is working on this, but I have given it some thought and I thought I'd share in case anyone else has some insight. First, some notes about template literals:
I am not super-familiar with this codebase but to me there are two main issues that need to be solved:
I think if we can solve problem 1 then problem 2 is straightforward: simply replace "$" with "\x24" when inserting into a template-literal context. In the regular javascript context (including the placeholder context) any data with a backtick would already have quotes wrapped around it so it would not be considered a template literal. Solving problem 1 is a harder given the approach of the existing code, which steps through the characters of the JS code but doesn't really parse the JS. In the code, the state of being in regular Javascript code is called I implemented the above as my initial solution, but then I realized it still isn't quite right, because the placeholder expression could contain "}"s. For example, At this point it seems to me that I am thinking of a level of complexity beyond how the JS-handing code currently works. So I'm wondering if I'm missing a simpler solution, or if these template literals complicate the situation enough that it is simply necessary to have a more complex Javascript "parser" for html/template to work with them. |
/cc @empijei |
Thanks @hundt for your analysis. It seems to me that fixing this issue would require this package to implement a JS parser to keep track of the current context. If not a fully-fledged JS parser it would at least need to correctly understand the context-switch between Template Literals and JavaScript context. While this is possible, it would complicate the current logic more than I'm comfortable with, and seems very tricky to get right. I played with this a little and stuff like the following line came up: var tmpl = `"${a="`${"}}`; My opinion would be to not support this and disallow JS template literals. This would need the documentation to be updated accordingly. I have some ideas on how to do this:
I would honestly already update the docs to say that we currently do not support template literals. We also need to decide what to do for cases like
Which are valid JS function calls. @mikesamuel do you have a different opinion? @hundt and @eternal-flame-AD what is the use case for JS template literals inside html templates? (I can guess hypothetical ones but I'm interested in your concrete one if you can publish it) |
A few notes:
I think this kind of thing is pretty well handled by the internal state machine once it understands the backticks, because it already knows that characters inside quotes lose the special meaning they have outside quotes. I tried this input with my initial implementation that I mentioned above, with some logging to indicate state transitions, and it did the right thing:
Here
I'm not sure it would be necessary to explicitly handle this. As long as backticks are implemented, I believe the current code will correctly understand that the "alert" part is in the normal Javascript context and we leave that context with the backtick.
Honestly in my case it was just a case of being lazy and using an embedded script with rendering logic inside of it on a page that also used html/template to inject in some server-side variables. My experience with this issue and my later reading of #27926 has convinced me that this was a bad approach, and I refactored my code to put substantially all of the Javascript into separately loaded modules which no longer pass through any server-side templating, which is what I probably should have done in the first place. Given that it is probably not good practice to do what I had been doing, in my opinion it would be reasonable not to support HTML templates with JS template literals (and fail when it encounters them), or to support it only with template literals formatted in the subset of possible ways that html/template can understand (provided it can reliably detect and error out when it encounters an unsupported situation). To me the biggest problem is that the API currently promises "safe" HTML but does not deliver it. Better to simply fail in that situation. In case it is of interest: I gathered that Google's closure-templates library is well-regarded, so I checked what they do. Just like html/template, they do a character-by-character "context" analysis of Javascript that falls short of fully parsing it. Their approach to template literals appears to be identical to the initial implementation I described above, including the failure to deal with braces inside |
Do you want to share a link to the patched code?
I agree. Plus I see little to no reason to nest Go and JS templates. The solution I'm more inclined to implement and document is to just have a @hundt if we can get to an agreement on this would you like to work on a CL to address it? |
Ah, sorry, wasn't sure the right way to share it without requesting a PR. Here's a patch, does that work for you? https://gist.github.com/hundt/fc27e649a01722a8466987b0f102fe5d
I think if we are going to support template literals then we would need to support
I'm interested but I will likely not be available in the short term. After another day or so I will likely have little time to work on this for at least two weeks. But if you don't mind waiting I can probably pick it up then. |
In case it helps illustrate some of the issues, here are some test cases indicating potential failure modes that allow code injection: https://play.golang.org/p/61h5vn53A4H The current code fails the first, second, and fourth tests because it does not understand template literals. My patch fails the fifth test because it understands template literals but not braces inside |
Thanks for this. To be honest I'd be more in favor of not supporting template literals at all instead of just partially supporting them (as much as you patch cover a good share of cases, aborting on I was proposing to drop all If this is not a common use case and wouldn't be useful I would remove TL completely. The approach would then be to just filter them or bail out when we see a backtick outside of strings. |
Ah okay, I misunderstood your proposal. I'm not familiar enough with how browsers parse things to know how feasible it is to "drop all I think it may be better not to support template literals at all (i.e. fail when we encounter them). #27926 has kind of convinced me that anything that discourages people from passing JS of any complexity through html/template might be a good idea. To quote from that proposal:
This is an example of an issue that did not exist when html/template was written, arose years later, and then continued on unfixed for more than four additional years, potentially leaving users vulnerable during that time. So I think it is good evidence in favor of the argument quoted, and against encouraging people to rely on html/template to understand their Javascript. So for that reason I am tentatively in favor of just not adding support for new Javascript features like this one. (Edit: but I do think we should add code to detect them, or anything that looks like them, and fail the template compilation.) |
You have a good point. +1 on bailing out if we see a template literal. If @mikesamuel agrees we can start working on it. |
It can be done purely based on lexical analysis. let myRegExp = / `${2}`.length / // No interpolation here.
2;
let myNum = 10 / `${2}`.length / // An interpolation here
2;
i ++
/ `${2}`.length / // Interpolation here
['index']
i =
++ / `${2}`.length / // No interpolation here
['index']
So matching
I agree. I included it because (JS was very stable at that time) and I wanted to enable <script>intiialize({{ .InitialClientSideState }})</script>
Fwiw, +1 |
Yeah, that seems very useful! My issue as a user who was not familiar with the codebase was that it was not clear to me that I should stick to very simple things like that and not try to pass any non-trivial scripts through html/template.
Just to be clear, you're talking about failing for any JS with template literals, right? And the earlier part of your comment (about interpolation) was for my edification (thanks!) but not relevant to the solution that would be implemented? |
Hi all, I found that Go's template package seems to be treating <script>let u = `https://foo.com`;</script> into <script>let u = `https:</script> Here's a playground link for this: https://play.golang.org/p/dvW0mCby1ED. Is it intentional? |
Providing a backward-compatible fix to this issue has demonstrated to be harder than initially thought. There is a change that is being experimented but it has breaking potential and I'm trying to evaluate the benefits and the issues with it. Thanks for adding more issues that this is causing, it helps building a case to address this, even if it may break some existing code. |
As a consumer of html/template - I am fine for disallowing literals and that go would just return error on executing the template when it sees a backtick string literal. However it's not fine to produce an invalid javascript with no indication of error anywhere (other than broken JS in user's browser). |
@karelbilek Yes, I agree with you that having I think this would be a relatively simple fix and I would be willing to help with it, but @empijei has mentioned the issue of backwards compatibility, so I assume that is why progress has been slow. Certainly the approach of "fail upon encountering a template literal" would cause some existing, correct code to stop working, and unfortunately there would be no way to tell at compile time whether a given program is affected. |
Oh. I can see how go backwards compat promise can bite it sometimes. |
I think this is worth mentioning: this issue has been fixed in safehtml/template. That package is a fork of If you are interested in seeing the fix implementation you can take a look here. The overall fix was 45 lines (excluding tests, but still not huge) and addresses the concerns: it ensures all ES6 template strings that appear in a Go safehtml/template are balanced and do not contain template actions. Should we consider to do the same here and accept to break all existing users that rely on this flaw? @FiloSottile I know that there is an escape hatch for the Go 1 guarantee for security, do you think we have a strong enough case here to apply it? |
@empijei That implementation is broken in the same way as my implementation from #9200 (comment) and the one in closure-templates: it fails to distinguish between a closing brace that ends an interpolation and one that ends a code block. For example:
is considered not to be an unclosed template literal by that code. |
Right. I didn't notice that, good catch (I reported it to the maintainers of that package). |
A recap for this issue so far. We have several possibilities in order of backward-compatibility breakage and security improvement:
All of the last four would address the "url breaks if in backticks" case. Did I miss anything? |
To be a little more precise, your solution (4) would ban interpolation/placeholders inside template literals but would allow simple template literals (like |
Updated, thanks. |
I ran into this today, took me a while to figure it out since one template with javascript backticks was impacted and another was not. It would be great if the html/template documentation mentioned that javascript template strings aren't currently supported. I can open a PR for that if it's wanted. |
by opennota:
The text was updated successfully, but these errors were encountered: