String literal expressions

[mattheww]

This fills in most of what is still missing for #939.

I haven't tried to cover C string literals at this stage, as they haven't quite settled down yet.

I haven't removed much of the text in tokens.md which is discussing what string literal tokens "represent" or "denote", although I think strictly speaking that applies to literal expressions rather than literal tokens.

I think the best course would be think about that after, or together with, making the Tokens chapter correct (which is #626).

[ehuss]

Perhaps to avoid confusion here?

Suggested change

	That is, the type is `&'static [u8; N]`, where `N` is the number of characters in the represented string described below.
	That is, the type is `&'static [u8; N]`, where `N` is the number of bytes in the represented string described below.

Also, this may become a little more confusing after RFC 3349 is implemented/stabilized.

[mattheww]

After RFC 3349 I think the byte and byte-string literals will need a different approach, with a separate version of the whole "Escapes" section. But that will be easier to describe when the new forms of escape are there.

[ehuss]

I think this is fine to keep. Just a note that my preference longer-term would be to have some global explanation of the CRLF→LF conversion in a single place so it doesn't need to be repeated in multiple places. For example, this is relevant for all string types (raw strings, byte strings, c-strings, etc.), but they don't currently mention this in the same way.

If there is a section somewhere in the lexing chapter that talks about this translation, I think these sentences could be removed.

[mattheww]

I'm thinking of looking at documenting CRLF conversion properly next (rather than trying to fight all of #626 in one go).

[ehuss]

Thanks!

I think it would be fine to also include the c-string section if you want. I believe the discussion has settled on that.

[mattheww]

I've force-pushed with your suggested changes.

I think I'd better look at C strings separately. I think I'm going to need some help with the "The expression's value is …" part.

[ehuss]

Thanks!