Skip to content

Commit

Permalink
Lexer.x: Fix regex lexing bug
Browse files Browse the repository at this point in the history
The simplest example of what was failing to compile was `/[/]/` where
the forward slash in the middle was being incorrectly detected as the
end of the regex.

Closes: #44
  • Loading branch information
erikd committed Mar 18, 2016
1 parent 47159a0 commit c5e0bce
Showing 1 changed file with 12 additions and 5 deletions.
17 changes: 12 additions & 5 deletions src/Language/JavaScript/Parser/Lexer.x
Expand Up @@ -90,8 +90,13 @@ $short_str_char = [^ \n \r ' \" \\]
-- $RegExpChars = [$printable] # [\\]
-- {Non Terminator} = {String Chars1} - {CR} - {LF}
-- $NonTerminator = $StringCharsDoubleQuote # [$cr $lf]
$NonTerminator = [$printable] # [$cr $lf]
-- {Non Zero Digits}={Digit}-[0]
$regNonTerminator = [$printable] # [$cr $lf \[]
$reg_char_class_chars = [^ $cr $lf \[ \] ]
@reg_char_class_escapes = \\ ( \[ | \] )
@regCharClass = \[ ($reg_char_class_chars | @reg_char_class_escapes)* \]
-- ~ (LineTerminator | MUL | BSLASH | DIV)
$RegExpFirstChar = [$printable] # [ $cr $lf \* \\ \/]
Expand Down Expand Up @@ -219,7 +224,7 @@ tokens :-
-- StringLiteral = '"' ( {String Chars1} | '\' {Printable} )* '"'
-- | '' ( {String Chars2} | '\' {Printable} )* ''
<reg,divide> $dq (@stringCharsDoubleQuote *) $dq
| $sq (@stringCharsSingleQuote *) $sq { adapt (mkString stringToken) }
| $sq (@stringCharsSingleQuote *) $sq { adapt (mkString stringToken) }
-- HexIntegerLiteral = '0x' {Hex Digit}+
<reg,divide> ("0x"|"0X") @HexDigit+ { adapt (mkString hexIntegerToken) }
Expand All @@ -230,8 +235,10 @@ tokens :-
-- RegExp = '/' ({RegExp Chars} | '\' {Non Terminator})+ '/' ( 'g' | 'i' | 'm' )*
-- <reg> "/" ($RegExpChars | "\" $NonTerminator)+ "/" ("g"|"i"|"m")* { mkString regExToken }

-- Based on the Jint version
<reg> "/" ($RegExpFirstChar | "\" $NonTerminator) ($RegExpChars | "\" $NonTerminator)* "/" ("g"|"i"|"m")* { adapt (mkString regExToken) }
<reg> "/"
("\" $regNonTerminator | @regCharClass | $RegExpFirstChar)
("\" $regNonTerminator | @regCharClass | $RegExpChars)* "/" ("g"|"i"|"m")* { adapt (mkString regExToken) }




Expand Down

0 comments on commit c5e0bce

Please sign in to comment.