Support escape characters in source code

[axic]

bytes memory x = "\x12\xbc"; or the same with string will fail with an error:

VM485:1 Uncaught SyntaxError: Unexpected token :
n.parseJSON @ jquery-2.1.3.min.js:4
compilationFinished @ index.html:537
compileJSON @ index.html:598
(anonymous function) @ index.html:532
gatherImports @ index.html:654
compile @ index.html:526

It compiles with solc, when not in the browser.

[chriseth]

Probably duplicate of #114

[axic]

This is part of the compiler output JSON:

{"attributes":{"string":"null","type":"literal_string \"\u0012琢�value":"\u0012ﵲ�ame":"Literal"}],"name":"VariableDeclaration"}

Probably jQuery's parseJSON can't properly handle those UTF8 fields?

Edit: that indeed looks a bit broken.

[axic]

For the input of bytes memory x = "\x12\x0c";, the appropriate JSON piece will be:

{"attributes":{"string":"null","type":"literal_string \"\u0012\f\"","value":"\u0012\f"},"name":"Literal"}

[chriseth]

What does the compiler do about var x = "\\x01";?

[axic]

It seems to be a double escape on both JS and the compiler as it remains a string of \\x01.

[axic]

@chriseth Did a bit more testing with this.

The output via solc (via -o . --ast-json):

000002c0  61 6d 65 22 20 3a 20 22  45 6c 65 6d 65 6e 74 61  |ame" : "Elementa|
000002d0  72 79 54 79 70 65 4e 61  6d 65 22 7d 2c 7b 22 61  |ryTypeName"},{"a|
000002e0  74 74 72 69 62 75 74 65  73 22 20 3a 20 7b 22 73  |ttributes" : {"s|
000002f0  74 72 69 6e 67 22 20 3a  20 22 6e 75 6c 6c 22 2c  |tring" : "null",|
00000300  22 74 79 70 65 22 20 3a  20 22 6c 69 74 65 72 61  |"type" : "litera|
00000310  6c 5f 73 74 72 69 6e 67  20 5c 22 5c 75 30 30 31  |l_string \"\u001|
00000320  32 80 5c 22 22 2c 22 76  61 6c 75 65 22 20 3a 20  |2.\"","value" : |
00000330  22 5c 75 30 30 31 32 80  22 7d 2c 22 6e 61 6d 65  |"\u0012."},"name|
00000340  22 20 3a 20 22 4c 69 74  65 72 61 6c 22 7d 5d 2c  |" : "Literal"}],|

This is the JSON generated by the ASTJsonConverter and the output seems invalid: \u0012 as a text followed 0x80 as a raw value.

The output via solcjs (saved the output of compileJSON directly to a file, without trying to JSON parse it):

00000df0  6e 67 22 7d 2c 22 6e 61  6d 65 22 3a 22 45 6c 65  |ng"},"name":"Ele|
00000e00  6d 65 6e 74 61 72 79 54  79 70 65 4e 61 6d 65 22  |mentaryTypeName"|
00000e10  7d 2c 7b 22 61 74 74 72  69 62 75 74 65 73 22 3a  |},{"attributes":|
00000e20  7b 22 73 74 72 69 6e 67  22 3a 22 6e 75 6c 6c 22  |{"string":"null"|
00000e30  2c 22 74 79 70 65 22 3a  22 6c 69 74 65 72 61 6c  |,"type":"literal|
00000e40  5f 73 74 72 69 6e 67 20  5c 22 5c 75 30 30 31 32  |_string \"\u0012|
00000e50  ef a9 8a ef bf bd 76 61  6c 75 65 22 3a 22 5c 75  |......value":"\u|
00000e60  30 30 31 32 ef b5 b2 ef  bf bd 61 6d 65 22 3a 22  |0012......ame":"|
00000e70  4c 69 74 65 72 61 6c 22  7d 5d 2c 22 6e 61 6d 65  |Literal"}],"name|

As it can be seen that non-Unicode/non-ASCII output gets distorted by either JSON::FastWriter in solc or by Emscripten.

80 gets turned into ef a9 8a ef bf bd, swallowing the subsequent " too

[axic]

argotorg/solidity#603 explains this issue at the root cause, closing this in favour of it.