Support escaped CR and CRLF in strings

[jordanbtucker]

Support escaped CR and CRLF in strings.
Add new peek method to facilitate look-aheads.

[jordanbtucker]

The multi-string test cases were failing on Windows due to CRLF line endings.

[aseemk]

@jordanbtucker: you are totally awesome! Thanks for so many great suggestions and pull requests. I'm glad @aeisenberg stepped in for one, but he's on vacation, and I'm super busy with work right now.

Given your awesome contributions so far, I'm going to make you a collaborator and give you push rights to this repo. Please do keep on opening pull requests and giving others a chance to review your changes, but you can now make the call to merge yourself. (E.g. for minor changes, or if no one has reviewed your code after some time.)

Welcome to the team! =)

[aseemk]

A few questions about this:

1. Why do you need to see whether a \n is the next character? Is \r by itself not enough to ignore?
2. Why is this within the if (ch === '\\') block? Isn't this looking at carriage returns only after an escaped backslash? Why not put this further down, next to the regular ch === '\n' check?

[aseemk]

Ah, ignore the second question; I understand. I forgot that our multi-line strings require a backslash escape, so this needs to come after that. =)

Still curious about the first q.

[aseemk]

And btw, if \r by itself should be accepted too, then you might not need this block of code at all (nor peek of course). See the escapee dictionary up top (if you haven't already) where \n is handled.

[jordanbtucker]

The reason I'm handling \r as a special case is due to the differences in how operating systems represent new lines.

• \n is used by Unix, Linux, and OS X and later.
• \r is used by OS 9 and earlier.
• \r\n is used by Windows (note that it is a \r followed by \n).

Therefore, any of the following combinations constitute an escaped new line:

• \ followed by \n
• \ followed by \r not followed by \n
• \ followed by \r\n

This conforms to the LineContinuation production, which uses the LineTerminatorSequence production.

There's no need to add \r to escapee since it's always handled as a special case.

[jordanbtucker]

I'm thinking I should create test cases for these before merging. Working on that now.

[aseemk]

I didn't mean why \r in general — yep, I'm aware of different line endings =D — I meant why are you peeking ahead in the \r case to look for an \n? Why not just ignore \r and \n each independently? It'll then work when they're back-to-back as well. And to ignore \r independent of \n, you would just add an entry for \r into escapees just like \n.

[jordanbtucker]

I don't think that will work. Supposed we added \r to escapee and removed its special case. Then we parse a string like this:

'line1 \\\r\n line 2'

When the parser hits the \ character, it will check for a following u, which won't be there. So it moves on to check for one of the escapees, finding \r. It appends an empty string to string, and moves on to the next iteration.

Now ch contains \n, but it doesn't know that there was a \ before the \r that was before this character. so it will hit the check for \n returning a Bad string error.

I ran some tests to confirm this too.

[aseemk]

Oh, got it! Duh, sorry about that. Thanks for explaining. =)

[aseemk]

Nice, both this and the editorconfig.

[aseemk]

Great test cases! 🚢