Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

SlimIt breaks on unicode byteorder marks #35

Open
pannal opened this Issue Sep 3, 2012 · 6 comments

Comments

Projects
None yet
3 participants

pannal commented Sep 3, 2012

Example: "jquery.blockUI.js" and "jquery.caret.min.js"

Both have the unicode byteorder mark "EF BB BF" before or after the first line of code. Result is:

Illegal character '\xbb' at 1:1 after LexToken(ID,'\xef',1,0)
Illegal character '\xbf' at 1:2 after LexToken(ID,'\xef',1,0)

and errors in the browser with compressed versions of those:
ERROR: ILLEGAL TOKEN

@ghost ghost assigned rspivak Sep 3, 2012

Owner

rspivak commented Sep 3, 2012

Hi,

Could you give me links to those particular files with BOMs?

pannal commented Sep 7, 2012

Well, essentially this isn't slimit's fault, though it shouldn't break as BOMs are valid (at least in the correct position) I guess.

Owner

rspivak commented Sep 8, 2012

I agree. It's not the slimit's fault per se, but I don't mind adding BOM cleanup to Slimit. And thanks for the above links.

Illegal character '\xbb' at 1:10 after LexToken(ID,'\xef',1,9)
Illegal character '\xbf' at 1:11 after LexToken(ID,'\xef',1,9)

аналогичная ошибка произошла при использовании в теле шаблона :

{% compress js %}
<script type="text/javascript">
[invisible symbol BOM]$(document).ready(function(){
/* ... */
</script>
{% endcompress %}

дело в том, что я скопировал script с web-страницы, где закрался спец-символ &#65279
Естественно в шаблоне его совсем не видно, поэтому я указал вместо него [invisible symbol BOM]
И такое видимо, случается, даже почитав этот топик не сразу понял где ошибка, помогло только попеременное включение скриптов.

git вдобавок порезал мой пост...

Еще один момент:

Unexpected token (DIV, '/') at 1:231862 between LexToken(COLON,':',1,231860) and LexToken(BXOR,'^',1,231863)

"regex":"/^[0-9-()\ ]+$/" // <<-- ERROR

решение:

"regex":"/^[0-9-()\u0020]+$/" // <<-- RESOLVE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment