Design string tokens

[matklad]

We currently have ast::String for "" strings and ast::RawString for r"". We don't have a type to represent either of those.

Steps:

1. Find out places in grammar where we need this. One is extern "ABI". Are there anything else?
2. If the answer is no, we should change the grammar of ABI to contain ast::Literal (like we do for attributes).
3. If the answer is yes, we should consider adding a StringOrRawString type, with a better name, and maybe re-design literal to account for that (and then we should also introduce ByteStringOrRawByteString).
4. Finally, for all places that can have strings (extern ABI), we should add the requisite accessor.

But lets not overthink this and figure out 1 first.

[frazar]

I don't know if this can help, but the list of valid ABI names for rustc is defined here.

[lnicola]

I don't know if this can help, but the list of valid ABI names for rustc is defined here.

I think this is more about where in the Rust syntax you can have " quoted strings, besides string literals, attributes and ABI names. I thought for a while about it and couldn't find another place.

So that would put us in the first case above:

If the answer is no, we should change the grammar of ABI to contain ast::Literal (like we do for attributes).

[matklad]

Now that I think about this... why do we have ast::String and ast::RawString in the first place? They seem to be syntactically the same, and should probably be just ast::String...