-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unicode escapes are ordinary escape sequences #8480
Merged
Merged
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
43 changes: 43 additions & 0 deletions
43
compiler/test/dotty/tools/dotc/parsing/StringInterpolationPositionTest.scala
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
package dotty.tools | ||
package dotc | ||
package parsing | ||
|
||
import ast.untpd._ | ||
import org.junit.Test | ||
|
||
class StringInterpolationPositionTest extends ParserTest { | ||
|
||
val tq = "\"\"\"" | ||
val program = s""" | ||
|class A { | ||
| val expr = 42 | ||
| val s0 = s"string1" | ||
| val s1 = s"string1$${expr}string2" | ||
| val s2 = s"string1$${expr}string2$${expr}string3" | ||
| val s0m = s${tq}string1${tq} | ||
| val s1m = s${tq}string1$${expr}string2${tq} | ||
| val s2m = s${tq}string1$${expr}string2$${expr}string3${tq} | ||
|}""".stripMargin | ||
|
||
@Test | ||
def interpolationLiteralPosition: Unit = { | ||
val t = parseText(program) | ||
t match { | ||
case PackageDef(_, List(TypeDef(_, Template(_, _, _, statements: List[Tree])))) => { | ||
val interpolations = statements.collect{ case ValDef(_, _, InterpolatedString(_, int)) => int } | ||
val lits = interpolations.flatten.flatMap { | ||
case l @ Literal(_) => List(l) | ||
case Thicket(trees) => trees.collect { case l @ Literal(_) => l } | ||
} | ||
for { | ||
lit <- lits | ||
Literal(c) = lit | ||
str <- List(c.value).collect { case str: String => str} | ||
} { | ||
val fromPos = program.substring(lit.span.start, lit.span.end) | ||
assert(fromPos == str, s"$fromPos == $str") | ||
} | ||
} | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1 @@ | ||
. // error: expected class or object definition | ||
|
||
\u890u3084eu // error: error in unicode escape // error: illegal character '\uffff' | ||
|
||
. // error: expected class or object definition |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,48 @@ | ||
-- Error: tests/neg/unicodeEscapes-interpolations.scala:2:27 ----------------------------------------------------------- | ||
2 | val badInters1 = s"foo \unope that's wrong" // error | ||
| ^ | ||
| invalid unicode escape at index 6 of foo \unope that's wrong | ||
-- Error: tests/neg/unicodeEscapes-interpolations.scala:3:32 ----------------------------------------------------------- | ||
3 | val badIntersEnd1 = s"foo \u12" // error | ||
| ^ | ||
| invalid unicode escape at index 8 of foo \u12 | ||
-- Error: tests/neg/unicodeEscapes-interpolations.scala:4:29 ----------------------------------------------------------- | ||
4 | val badInters3 = s"""foo \unope that's wrong""" // error | ||
| ^ | ||
| invalid unicode escape at index 6 of foo \unope that's wrong | ||
-- Error: tests/neg/unicodeEscapes-interpolations.scala:5:28 ----------------------------------------------------------- | ||
5 | val caretPos1 = s"foo \u12x3 pos @ x" // error | ||
| ^ | ||
| invalid unicode escape at index 8 of foo \u12x3 pos @ x | ||
-- Error: tests/neg/unicodeEscapes-interpolations.scala:6:34 ----------------------------------------------------------- | ||
6 | val caretPos2 = s"foo \uuuuuuu12x3 pos @ x" // error | ||
| ^ | ||
| invalid unicode escape at index 14 of foo \uuuuuuu12x3 pos @ x | ||
-- Error: tests/neg/unicodeEscapes-interpolations.scala:7:30 ----------------------------------------------------------- | ||
7 | val caretPos3 = s"""foo \u12x3 pos @ x""" // error | ||
| ^ | ||
| invalid unicode escape at index 8 of foo \u12x3 pos @ x | ||
-- Error: tests/neg/unicodeEscapes-interpolations.scala:8:36 ----------------------------------------------------------- | ||
8 | val caretPos4 = s"""foo \uuuuuuu12x3 pos @ x""" // error | ||
| ^ | ||
| invalid unicode escape at index 14 of foo \uuuuuuu12x3 pos @ x | ||
-- Error: tests/neg/unicodeEscapes-interpolations.scala:10:53 ---------------------------------------------------------- | ||
10 | val badIntersmultiAfter = s"foo $placeholder bar \unope that's wrong" // error | ||
| ^ | ||
| invalid unicode escape at index 7 of bar \unope that's wrong | ||
-- Error: tests/neg/unicodeEscapes-interpolations.scala:11:37 ---------------------------------------------------------- | ||
11 | val badIntersmultiBefore = s"foo \unope $placeholder that's wrong" // error | ||
| ^ | ||
| invalid unicode escape at index 6 of foo \unope | ||
-- Error: tests/neg/unicodeEscapes-interpolations.scala:12:56 ---------------------------------------------------------- | ||
12 | val badInterstmultiAfter = s"""foo $placeholder bar \unope that's wrong""" // error | ||
| ^ | ||
| invalid unicode escape at index 7 of bar \unope that's wrong | ||
-- Error: tests/neg/unicodeEscapes-interpolations.scala:13:40 ---------------------------------------------------------- | ||
13 | val badInterstmultiBefore = s"""foo \unope $placeholder that's wrong""" // error | ||
| ^ | ||
| invalid unicode escape at index 6 of foo \unope | ||
-- Error: tests/neg/unicodeEscapes-interpolations.scala:14:29 ---------------------------------------------------------- | ||
14 | val badInterother = s"this \p ain't legal either" // error | ||
| ^ | ||
|invalid escape '\p' not one of [\b, \t, \n, \f, \r, \\, \", \', \uxxxx] at index 5 in "this \p ain't legal either". Use \\ for literal \. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
object Example { | ||
val badInters1 = s"foo \unope that's wrong" // error | ||
val badIntersEnd1 = s"foo \u12" // error | ||
val badInters3 = s"""foo \unope that's wrong""" // error | ||
val caretPos1 = s"foo \u12x3 pos @ x" // error | ||
val caretPos2 = s"foo \uuuuuuu12x3 pos @ x" // error | ||
val caretPos3 = s"""foo \u12x3 pos @ x""" // error | ||
val caretPos4 = s"""foo \uuuuuuu12x3 pos @ x""" // error | ||
val placeholder = "place" | ||
val badIntersmultiAfter = s"foo $placeholder bar \unope that's wrong" // error | ||
val badIntersmultiBefore = s"foo \unope $placeholder that's wrong" // error | ||
val badInterstmultiAfter = s"""foo $placeholder bar \unope that's wrong""" // error | ||
val badInterstmultiBefore = s"""foo \unope $placeholder that's wrong""" // error | ||
val badInterother = s"this \p ain't legal either" // error | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
-- Error: tests/neg/unicodeEscapes.scala:3:25 -------------------------------------------------------------------------- | ||
3 | val badsingle = "foo \unope that's wrong" // error | ||
| ^ | ||
| invalid character in unicode escape sequence | ||
-- Error: tests/neg/unicodeEscapes.scala:4:26 -------------------------------------------------------------------------- | ||
4 | val caretPos = "foo \u12x3 pos @ x" // error | ||
| ^ | ||
| invalid character in unicode escape sequence | ||
-- Error: tests/neg/unicodeEscapes.scala:5:33 -------------------------------------------------------------------------- | ||
5 | val caretPos2 = "foo \uuuuuuu12x3 pos @ x" // error | ||
| ^ | ||
| invalid character in unicode escape sequence | ||
-- Error: tests/neg/unicodeEscapes.scala:6:29 -------------------------------------------------------------------------- | ||
6 | val carPosTerm = "foo \u123" // error | ||
| ^ | ||
| invalid character in unicode escape sequence | ||
-- Error: tests/neg/unicodeEscapes.scala:7:30 -------------------------------------------------------------------------- | ||
7 | val halfAnEscape = "foo \u12" // error | ||
| ^ | ||
| invalid character in unicode escape sequence | ||
-- Error: tests/neg/unicodeEscapes.scala:8:30 -------------------------------------------------------------------------- | ||
8 | val halfAnEscapeChar = '\u45' // error | ||
| ^ | ||
| invalid character in unicode escape sequence | ||
-- Error: tests/neg/unicodeEscapes.scala:9:29 -------------------------------------------------------------------------- | ||
9 | val `half An Identifier\u45` = "nope" // error | ||
| ^ | ||
| invalid character in unicode escape sequence |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
|
||
object Example { | ||
val badsingle = "foo \unope that's wrong" // error | ||
val caretPos = "foo \u12x3 pos @ x" // error | ||
val caretPos2 = "foo \uuuuuuu12x3 pos @ x" // error | ||
val carPosTerm = "foo \u123" // error | ||
val halfAnEscape = "foo \u12" // error | ||
val halfAnEscapeChar = '\u45' // error | ||
val `half An Identifier\u45` = "nope" // error | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
processed...OK | ||
unprocessed...OK | ||
after backslashes | ||
List(\, \, u, 0, 0, 4, 0) | ||
List(\, u, 0, 0, 4, 0) | ||
List(\, \, u, 0, 0, 4, 0) | ||
List(\, u, 0, 0, 4, 0) | ||
List(", (, [, ^, ", \, x, 0, 0, -, \, x, 1, F, \, x, 7, F, \, \, ], |, \, \, [, \, \, ', ", b, f, n, r, t, ], |, \, \, u, [, a, -, f, A, -, F, 0, -, 9, ], {, 4, }, ), *, ") | ||
List(b, a, d, \) |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was following up that the unicodes on line 11 were edited out, when I noticed that this (edit: partially) reverts my last change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The test has to assert because there is no longer a check file -- vulpix only checks the output if there is a check file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll see if I can restore those changes. Does vulpix detect a test failing an assertion? I guess I'll jiggle with it a bit and let you know.
As for the identifier on line 11, the variant that is still supported is in tests/run/unicodeEscapes.scala line 13-14, which makes good on the promise of temporarily.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
NBD, I PR'd it, thanks. I was about to futz with more parsing. It's amazing how quickly I forgot like how do I even run a test?