move triple-quoted string processing into parser

[nolta]

Strip out triple-quoted string indentation in the parser, instead of using the mstr macro.

Also, no longer treat tabs as 8 spaces. Instead, remove the longest common prefix of spaces and tabs.

Fixes #2682

[StefanKarpinski]

:yay:

[ScottPJones]

I wish I knew you were working on this. I ended up wasting time then.

[ScottPJones]

Is this a typo here? This is changing it from picking up the 2nd element of str (cadr str), to picking up the first (car str).

[nolta]

https://github.com/JuliaLang/julia/pull/11815/files#diff-841413a6d431c3fda7ec5dd7258d40f6L1670

[ScottPJones]

Over terseness can be an impediment to communication. In-line comments explaining the structure that you are processing might have made the change clear (not just for me, but for anybody else who needs to maintain this in the future).

[ScottPJones]

What does this do when there is a there is no common prefix?
The old code in Base.triplequoted didn't look for a prefix, the code I wrote in #11719 doesn't modify the string at all if there is no common prefix (like Python's textwrap.dedent, which seemed to be the inspiration for """ in julia).
How does this treat totally blank lines? The old code ignored them when calculating the leading indentation, I had to treat them specially, so that if they were followed by interpolation, they were still counted when calculating the common prefix, but ignored when not, so things like:

   """
   a
   $var

   b"""

work correctly. The interpolation part may not be an issue, because this code is happening before the code splits the string up into a vector, however the ignoring the indentation of totally blank lines still needs to be handled.
It will be nice to have this in the parser!

[tkelman]

cancelled appveyor, it will probably time out due to #11818 - we can restart the build later if that gets fixed, though there's likely to be a worse-than-usual backlog for the next few days

[nolta]

What does this do when there is a there is no common prefix?

Nothing.

How does this treat totally blank lines?

julia/test/strings.jl

Lines 895 to 900 in 5651bba

	@test """
	a
	b
	c
	""" == "a$(nl)b$(nl)$(nl)c$(nl)"

[ScottPJones]

What does this do when there is a there is no common prefix?

Nothing.

Good, that was one of the problems I was trying to fix, to make it act like Python's dedent in that case.
Good also about the blank lines... I wasn't ready to tackle fixing the issue in the parser like you did, my Scheme skills are decades rusty... nice to see it handled better.

[StefanKarpinski]

I think this is fine to merge as-is, but I wonder if we shouldn't do those tests by calling parse on strings – it seems quite likely that the test files will get whitespace cleaned at some point, which would rather confusingly break the tests.