Syntax Error: Invalid character \0129 (riot+compiler.min) #1588

Closed
sebastialonso opened this Issue Feb 11, 2016 · 16 comments

Projects

None yet

4 participants

@sebastialonso

More a warning than an issue, but I'd hate make people waste hours of their time like me.
I was using the riot+compiler.min.js last version. I was implementing a hierarchy of tags, with a parent tag as the manager of actions.

Because of this, I made heavy use of this.update() for the tag that received data from the others and needed to update its UI. This function triggered the weird error in the title and NO error trace (but I was running webkit on PySide, maybe that's on me).
Changing the minified version for the non-minified version fixed everything.

@aMarCruz
Member

@sebastialonso ,
I can't find that sequence in riot+compiler.min.js... can you reproduce the issue in plunker? please.

In the meanwhile you can re-compress riot with uglifyjs dist/riot/riot+compiler.js --comments -o dist/riot/riot+compiler.min.js , as the option --mangle used in our Makefile cause these problems sometimes.

Edit: Sorry, --mangle does not handle the character transformation. I'm looking for more info in the unglify repo.

@aMarCruz aMarCruz added a commit to riot/tmpl that referenced this issue Feb 12, 2016
@aMarCruz aMarCruz v2.3.22
It has changed the character used to hide quoted strings and regexes, maybe this fix [riot#1588](riot/riot#1588) : Syntax Error: Invalid character `\0129` (riot+compiler.min).
d37fe2f
@aMarCruz aMarCruz added a commit to riot/compiler that referenced this issue Feb 12, 2016
@aMarCruz aMarCruz v2.3.23
- It has changed the character used to hide expressions during the compilation, maybe this fix [riot#1588](riot/riot#1588) : Syntax Error: Invalid character `\0129` (riot+compiler.min).
- Update devDependencies
3259d56
@aMarCruz
Member

@sebastialonso , there's new versions in the "dev" branch of the compiler and tmpl that can fix this issue, uglify does not has an option to avoid the convertion of control characters.

It is an interesting problem... If this is how I think, this may be related to the way in which JS parses the strings :)

@sebastialonso

Indeed, it's a very interesting problem. In the meantime I have no issues using the normal version of Riot. It was incredibly challenging to debug :). I'll try to make another plnkr, the one I made wasn't able to reproduce the weird behavior.

I'm really glad I'm still a fan of Riot after all this. It's an awesome project!!

@aMarCruz
Member

@sebastialonso thank you :)

@Conduitry

If I'm understanding the issue here correctly - uglify-js's AST print_to_string() method has an option called ascii_only that I think might be another way to address this. https://github.com/mishoo/UglifyJS2/blob/master/lib/output.js

@aMarCruz
Member

@Conduitry thanks, ascii_only does not work with the compressor. I already checked it.

@aMarCruz aMarCruz added a commit to riot/compiler that referenced this issue Mar 8, 2016
@aMarCruz aMarCruz v2.3.23
- Fix [riot#1325](riot/riot#1325) : Gulp + Browserify + Babelify + type="es6" error.
- Fix [riot#1342](riot/riot#1342), [riot#1636](riot/riot#1636) and request from [dwyl/learn-riot#8](dwyl/learn-riot#8) : Server-Side Rendered Page Fails W3C Check. The new `data-is` attribute is used for scoped styles in addition to `riot-tag` (the later will be removed in compiler v3.x)
- The keyword `defer` in `<script src=file>` avoids that the compiler loads the file, preserving the tag - Requested by [riot#1492](riot/riot#1492) : Stop script tags from being evaluated with serverside `riot.render`. It is removed in client-side compilation because browsers will not load scripts through innerHTML.
- It has changed the character used to hide expressions during the compilation, maybe this fix [riot#1588](riot/riot#1588) : Syntax Error: Invalid character `\0129` (riot+compiler.min).
- Removed the unused parameter with the compiled-time brackets from the call to `riot.tag2`.
- Removed support for raw expressions. It is unlikely this feature will be implemented in v2.3.x
- Update devDependencies.
ba1f912
@aMarCruz aMarCruz referenced this issue in riot/compiler Mar 9, 2016
Merged

v2.3.23 #59

@aMarCruz aMarCruz added a commit to riot/compiler that referenced this issue Mar 10, 2016
@aMarCruz aMarCruz v2.3.23
- The parsers are moved to its own directory in the node version. The load is on first use.
- Fix [riot#1325](riot/riot#1325) : Gulp + Browserify + Babelify + type="es6" error.
- Fix [riot#1342](riot/riot#1342), [riot#1636](riot/riot#1636) and request from [dwyl/learn-riot#8](dwyl/learn-riot#8) : Server-Side Rendered Page Fails W3C Check. The new `data-is` attribute is used for scoped styles in addition to `riot-tag` (the later will be removed in compiler v3.x)
- The keyword `defer` in `<script src=file>` avoids that the compiler loads the file, preserving the tag - Requested by [riot#1492](riot/riot#1492) : Stop script tags from being evaluated with serverside `riot.render`. It is removed in client-side compilation because browsers will not load scripts through innerHTML.
- It has changed the character used to hide expressions during the compilation, maybe this fix [riot#1588](riot/riot#1588) : Syntax Error: Invalid character `\0129` (riot+compiler.min).
- The option `debug` inserts newlines between the `riot.tag2` parameters and the call is prefixed with the source filename - Requested by [riot#1646](riot/riot#1646) : Split portions of generated html with newline instead of space
- Removed the unused parameter with the compiled-time brackets from the call to `riot.tag2`.
- Removed support for raw expressions. It is unlikely this feature will be implemented in v2.3.x
- Updated the regex that is used to match tag names, more closer to the HTML5 specs.
- Update devDependencies.
f455dfd
@aMarCruz aMarCruz added a commit to riot/tmpl that referenced this issue Mar 10, 2016
@aMarCruz aMarCruz v2.3.22
- It has changed the character used to hide quoted strings and regexes, maybe this fix [riot#1588](riot/riot#1588) : Syntax Error: Invalid character `\0129` (riot+compiler.min).
- Removed support for raw expressions. It is unlikely this feature will be implemented in v2.3.x
- Update devDependencies
439b358
@aMarCruz aMarCruz added a commit to riot/tmpl that referenced this issue Mar 10, 2016
@aMarCruz aMarCruz v2.3.22
- It has changed the character used to hide quoted strings and regexes, maybe this fix [riot#1588](riot/riot#1588) : Syntax Error: Invalid character `\0129` (riot+compiler.min).
- Removed support for raw expressions. It is unlikely this feature will be implemented in v2.3.x
- Update devDependencies
f0485fa
@Conduitry

This issue, or a very similar one, seems to have resurfaced in 2.3.18 for me. The character \u2057 is getting minified to the actual unicode character in riot.min.js, and this is causing a syntax error for me, possibly because I'm not setting the correct encoding headers on the javascript. If I replace the single character in the minified source with \u2057, all is fine. I did not have this trouble with 2.3.17.

@Conduitry

The culprit looks to be this guy https://github.com/riot/tmpl/blob/master/src/tmpl.js#L160

  var
    CH_IDEXPR = '\u2057',
    RE_CSNAME = /^(?:(-?[_A-Za-z\xA0-\xFF][-\w\xA0-\xFF]*)|\u2057(\d+)~):/,
    RE_QBLOCK = RegExp(brackets.S_QBLOCKS, 'g'),
    RE_DQUOTE = /\u2057/g,
    RE_QBMARK = /\u2057(\d+)~/g

UglifyJS isn't minifying the \u2057 that appears in the regular expression, so replacing CH_IDEXPR's definition with something like RE_DQUOTE.source should work, although it wouldn't be very pretty.

@GianlucaGuarini
Member

It's strange because riot-tmpl did not change between 2.3.17 and 2.3.18

@Conduitry

The code in riot-tmpl that I'm having trouble with appeared in riot-tmpl 2.3.22. It looks like riot 2.3.17 depended on riot-tmpl ^2.3.21, but the bundled riot.js (and riot.min.js) did not use 2.3.22, they used 2.3.21. See https://github.com/riot/riot/blob/2684fa8a644ebc78f4f3f6a0796e000faf9cc1cc/riot.js#L761

@GianlucaGuarini
Member

@Conduitry you are right @aMarCruz can you have a look at this issue please?

@aMarCruz
Member
aMarCruz commented Apr 7, 2016

@GianlucaGuarini yes, I will try with .source.

@aMarCruz aMarCruz self-assigned this Apr 7, 2016
@aMarCruz
Member

@Conduitry is this issue solved in the current riot version?

@Conduitry

@aMarCruz In 2.5.0? No, it's not.

https://raw.githubusercontent.com/riot/riot/v2.5.0/riot.min.js This still has the non-ascii character from the uglifyjs minification.
https://raw.githubusercontent.com/riot/riot/v2.5.0/riot.js This still has CH_IDEXPR = '\u2057' in it - it's not using RE_DQUOTE.source.

@aMarCruz
Member
aMarCruz commented Aug 30, 2016 edited

@Conduitry , replacing CH_IDEXPR with RE_DQUOTE.source breaks the parsing.
CH_IDEXPR is the character '\u2057', it is not the JS string "\u2057".

I will push a new version in the "dev" branch which set CH_IDEXPR with String.fromCharCode(0x2057)

@aMarCruz aMarCruz added a commit to riot/tmpl that referenced this issue Aug 30, 2016
@aMarCruz aMarCruz Possible fix to riot/riot#1588 38db8e4
@Conduitry

@aMarCruz Ah oops true, I wasn't thinking about that. Just using String.fromCharCode sounds good then.

@aMarCruz aMarCruz referenced this issue in riot/tmpl Oct 13, 2016
Merged

Merge Dev into Master #18

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment