Regular expression parse error - /^\_/ and /\\{/g

[hyunjunekim]

Hi, I have some tests for regular expression.

case 1

var s = "abcd";
s.search(/^\_/);

SyntaxError: invalid regexp escape (line 1)
    duk_lexer.c:1578

case 2

var s = "abcd";
s.search(/\\{/g);

SyntaxError: invalid regexp quantifier (unknown char) (line 1)
    duk_lexer.c:1468

In the regular expression, I think that '' has some problem.

[svaarala]

The Ecmascript specification is quite strict in what escapes are allowed and required. In your case:

• Case 1: it's not valid to escape an underscore, so the correct form would be /^_/, without the underscore.
• Case 2: an open curly brace begins a quantifier of the form {n} or {n,m}. If you want to match a literal open curly brace, it must be escaped: /\\\{/.

Most Ecmascript engines are rather loose about regexp syntax, accepting many regexps that are technically syntax errors in the specification (as far as I've been able to determine). Because this is common behavior, Duktape may need to move into that direction, but right now it follows the E5 specification quite strictly.

[hyunjunekim]

@svaarala
Case 1 : I think that it's valid.
2.8.7 Regular Expression Literals

_ = > _( \ RegularExpressionNonTerminal )
_ => SourceCharacter
So

/^\_/

is possible.

[svaarala]

Ecmascript specification defines RegExp syntax in Section 15.10.1:

• http://www.ecma-international.org/ecma-262/5.1/#sec-15.10.1

Ultimately it comes down to IdentityEscape:

IdentityEscape ::
SourceCharacter but not IdentifierPart
<ZWJ>
<ZWNJ>

The escaped character cannot be an identifier part, which includes a-z, A-Z, underscore, etc.

[hyunjunekim]

IdentifierPart ::
IdentifierStart
UnicodeCombiningMark
UnicodeDigit
UnicodeConnectorPunctuation
<ZWNJ>
<ZWJ>

Because of that '_' is IdentifierStart, Is Case1 invalid?
And What do you mean "realworld"label ?

[svaarala]

Yes, IdentifierPart includes all IdentifierStart characters, e.g. underscore.

The "realworld" label means that the desired behavior is not compliant but expected by users.

[hyunjunekim]

@svaarala Thank you. :)

[svaarala]

I might be wrong in my interpretation of the RegExp syntax in E5 - but so far I haven't been informed otherwise :)

[hyunjunekim]

@svaarala
In the chrome for V8, very well for case1 and case2.
Is there a difference between Ecmascript Regex and Javascript Regex?

[hyunjunekim]

@svaarala
special char('_' or '[' or ']' etc), need '' backslash.

[svaarala]

No, as far as I understand V8 and other engines don't comply to the E5 specification - they have a wider regexp syntax.

[sva-p]

Underscore ('_') is not a special character and should not require escaping.

[hyunjunekim]

Sorry, very well for '/a[0]/'

[svaarala]

You were missing the trailing trailing slash: /a\[0\]/ if you want to match the brackets literally.

[hyunjunekim]

@svaarala
I learned a lot for you. Thank you. :)

[svaarala]

No problem :)

Anyway, as I said, it's somewhat confusing that Ecmascript engines vary between what they accept and reject for regexps.

[hyunjunekim]

@svaarala
I understand that duktape is strict for Regex rather than other engines. :)

[svaarala]

Since there's nothing to fix in this issue, I'll close this one. I added #74 to track all the known regexp "real world" issues so that there's a clear idea of what cases will at least need fixing.