Ignore tokens in RegExp

[shuhei]

Some tokens in RegExp literal caused the following problems:

• Unclosed quotes mess up the state so that following require() calls are not properly detected. e.g. var pattern = /"/; require("a"); then a is not detected.
• require() calls in RegExp literal are wrongly detected. e.g. var pattern = /require("a")/; then a is detected.

To fix those issues, this patch treats RegExp literals as strings with slash as quote. To keep supporting shebang that inevitably contains slashes, now it is handled before the parsing starts.

[shuhei]

Uh, no. This patch can't distinguish between the start of regexp and the division operator...

[shuhei]

Revised the code to naively distinguish between regexp leteral and division operator. Now it assumes it's regexp literal if preceding token is one of (,=:[!&|?{};. But it makes mistakes for some cases like the following:

if (something) /'/.exec(x);
require("a");

return /"/.test(x);
require("b");

Then a or b won't be detected. Please check out a discussion on Stack Overflow](http://stackoverflow.com/questions/5519596/when-parsing-javascript-what-determines-the-meaning-of-a-slash) for more detail.

Should we do more complicated things to support those cases or keep this parser simple and just working for most of the cases?

[glenjamin]

I was having a go at fixing this, had something working before noticing your open PR.

My first stab also missed the division case entirely - although I did cover escape sequences in regexp literals.

I've pushed my work, including a whole bunch of failing scenarios
https://github.com/glenjamin/mine/blob/regex/test/files/regex.js

I've been using http://www.antlr3.org/grammar/1153976512034/ecmascriptA3.g and http://tomcopeland.blogs.com/EcmaScript.html for reference.

If you use the latter and start at http://tomcopeland.blogs.com/EcmaScript.html#prod68 clicking the left-most non-terminal you end up with the literal regex.
http://tomcopeland.blogs.com/EcmaScript.html#prod29 covers the division case
http://tomcopeland.blogs.com/EcmaScript.html#prod9 and http://tomcopeland.blogs.com/EcmaScript.html#prod59 cover the division assignment case

In theory we should be able to use these grammars to figure out the exact subset needed to reliably extract all the require() calls - or maybe we do actually need everything? If that's the case something based on http://zaach.github.io/jison/docs/ might be more apt.

Anyway, that's my 2¢ after spending the evening trying to figure out why i couldn't browserify some modules!