📍 Adding here strings

[Angelo1211]

So these work similar to the #string directives in the language, but use a single character as a delimiter rather than an identifier.

There are two issues I see as blockers to merging this, but wanted to open this pr to discuss:

• When escaping the delimiter within the string you will keep the \ in the original string, which is ugly and likely not what the user was expecting to happen. We could deal with this when copying the string - if the user requested that -, but that's more complexity and doesn't really solve the non-copying case. We might have to get more creative with how to do the escape or revert back to the idea we discussed of having some kind of symmetric syntax with the @ like so:

s = @"  "As it is I don't see the point otherwise" "@

• We will unparse these strings using regular "" qouted strings. If you've parsed and then unparsed the string it might be a problem, as you'd lose the nice presentation you might have worked really hard for in your string. Not quite sure how to deal with this one at the moment!

[osor-io]

The first issue already happens with normal strings. If you do data = "This data has one of these \", oh no" you will get back the string with the \" in it (meaning not a single "). So that we probably shouldnt' consider a blocker.

For the second thing, it's tricky. When we parse the data it becomes a normal data type for the user and it probably should be kept that way. We could support some sort of annotation on string parameters that write them back with a custom delimiter on unparse(), something like the following.

My_Data :: struct
{
    text : string; @multiline"
}

Maybe this is doable with custom_struct_unparsing() but then you still have the issue of reading in the string. If you do custom unparsing you're probably expecting to do custom parsing as well. Which in this case can't happen because this is at the tokenizer level.

In general though, I don't think people should have the full expectation that unparse is going to keep things like their original formatting in the file. Obviously stuff like indentation is gonna change, and if you change values to their default value they won't be output in the current unparse() code.

So I think there's a way to get this in with the annotation. But conceptually this also feels like something that should be doable via some custom code and it isn't at the moment 🤔 Perhaps the way is to allow for custom tokenizer code to be provided as well. Then you could use any of the symbols as the start of a multiline string, and when unparsing, the custom unparse code would handle it.