Tagged strings

[munificent]

This is the tracking issue for the tagged strings language proposal.

[ykmnkmi]

it's executed in compile time? can it be const?

const List<int> http = ascii'HTTP'; //
while (stringScanner.readChar() != c'\n') {}

[Levi-Lesches]

I didn't see anything in the proposal about it being executed in compile-time, just that it's desugared into a function call, so no.

[mraleph]

I have some concerns about performance / code size implications associated with this mechanism as it is proposed: there is an invisible cost (both in terms of performance and code size) associated with each subexpression ($expr) due to implicit wrapping of it in a closure.

[Quijx]

Is this proposal a replacement/evolution of #1479? To me it looks like almost the same expressive power with the same syntax, just a different way to define the tag processor/string interpolator.

[Quijx]

As a possible solution to the async problem I propose that the way the tagged string is desugared depends on the type signature of the tag processor.

For example here some desugarings for the tagged string foo 'bar${await baz()}';:

// For
Foo fooStringLiteral(List<String> strings, List<Object? Function()> values) { ... }
// the tagged string is desugared to
final temp = await baz();
fooStringLiteral(['bar'],[() => temp]);

// For
Foo fooStringLiteral(List<String> strings, List<Object?> values) { ... }
// the tagged string is desugared to
final temp = await baz();
fooStringLiteral(['bar'],[temp]);

// For
Foo fooStringLiteral(List<String> strings, List<FutureOr<Object?> Function()> values) { ... }
// the tagged string is desugared to
fooStringLiteral(['bar'],[() async => await baz()]);

// For
Foo fooStringLiteral(List<String> strings, List<FutureOr<Object?>> values) { ... }
// the tagged string is desugared to
fooStringLiteral(['bar'],[(() async => await baz())()]);
// or
fooStringLiteral(['bar'],[baz()]);
// which should be the same

This would also solve the performance problems that @mraleph mentioned because if the tag processor does not lazily evaluate its interpolated values, the signature can just accept the values directly and no wrapping closures are created.

Also to would be nice if the Type of the interpolated values could be restricted so that for Foo fooStringLiteral(List<String> strings, List<Bar> values) { ... }, the tagged string foo 'bar${1}'; would not compile because 1 is not of type Bar.

[lrhn]

If interpolation expressions are definitely thunkified, then they are not necessarily executed in order.
That makes some cases harder to handle, like promotion in one expression affecting or not affecting following expressions.
Not sure how of then that would happen in practice (I can't find a reasonable example).

It also means that any variable mentioned in an interpolation expression is effectively inside a function expression, and that also affects how we can promote the variable outside of the string literal. That's worse.

Making it depend on the type of tag's value parameter is possible, but also extremely opaque.
Could a tag have multiple handlers for different interpolation expressions of different types?

(See also #1479, which I prefer not just because I wrote it, but because it makes the order of interpolations vs string slices explicit in the call order, and avoids allocating extra lists that might not be necessary).

[munificent]

I have some concerns about performance / code size implications associated with this mechanism as it is proposed: there is an invisible cost (both in terms of performance and code size) associated with each subexpression ($expr) due to implicit wrapping of it in a closure.

Yes. From talking to @jakemac53 and discussion on #1983, I've been convinced that they should be eagerly evaluated. I could see us adding something like late parameters as a separate language feature (and then allowing the parameters to the tagged string processor to use it) in order to opt in to lazy evaluation, but it feels weird to make it the default here when no other place in the language lets you do this.

As a possible solution to the async problem I propose that the way the tagged string is desugared depends on the type signature of the tag processor.

This is a good suggestion. I did consider this but overall we try to mostly avoid having language semantics hang directly off the static types of parameters. (Context types do affect semantics in a few cases, but rarely in ways that feel like a runtime behavioral difference.) It gets really weird if you consider code like:

T fooStringLiteral<T>(List<String> strings, List<T> values) { ... }

Dart doesn't do specialization, so it would be very strange if some invocations of this tagged string were lazy and some weren't based on the type parameter that got inferred.

Also to would be nice if the Type of the interpolated values could be restricted so that for Foo fooStringLiteral(List<String> strings, List<Bar> values) { ... }, the tagged string foo 'bar${1}'; would not compile because 1 is not of type Bar.

Good suggestion. I'm going to update the proposal with that shortly. :)

[stereotype441]

If interpolation expressions are definitely thunkified, then they are not necessarily executed in order. That makes some cases harder to handle, like promotion in one expression affecting or not affecting following expressions. Not sure how of then that would happen in practice (I can't find a reasonable example).

Ooh, the promotion issue definitely worries me. It means things like this wouldn't work in an intuitive way:

f(int? x) {
  if (x != null) {
    foo 'magic interpolation involving ${x + 1}'; // (1)
  }
  x = null; // (2)
}

(Flow analysis would have to treat the subexpression x + 1 at (1) as appearing inside a closure, and since it has no way to guarantee that the closure will be called before (2), this would be an error since x can be null).

[munificent]

That's an excellent point. I updated the proposal yesterday to state that the expressions are eagerly evaluated. This more firmly convinces me that eager is the right answer.

[Quijx]

Is this feature still being pursued even if the metaprogramming does not end up needing it? I feel like this could still be a very usefull featue in any case.

For example I am dreaming of a world where LaTeX is replaced by a dart framework where text in strings can be combined with widget-like components for graphics, styling, etc. using the tagged strings feature. Also darts hot reload could be used to support incremental compilation of the pdf, which could be much faster for large files than compiling everything from scratch (like LaTeX does). So I think dart would be a good match for something like this and this feature could open the door for that.
One can dream, right? 😁

[munificent]

Is this feature still being pursued even if the metaprogramming does not end up needing it? I feel like this could still be a very usefull featue in any case.

I agree that it's still a useful feature even without metaprogramming. (I wrote an internal proposal for it many many years ago well before we ever thought about macros.) But without being used there, the priority and value of the feature goes down. I don't know if it would be useful enough to prioritize it over other features at that point.

[Leedehai]

Hi - it's been almost a year since the wonderful proposal.

I'm wondering if this feature is still being considered by the team, or is it safe to consider it as having been indefinitely shelved? Thanks 😊

[munificent]

It's not being actively worked on right now. We've got our hands full with views, records, patterns, and some other work. :)

[bivens-dev]

Wondering if this is anything you might be reconsidering now after shipping 3.0? I’d love to be able to use something like this to generate my server side HTML responses.

[munificent]

I'm still interested in the proposal (and think it would be helpful for macro authors to make a nicer API for generating code), but it hasn't risen to the top of the priority list yet.

[bivens-dev]

It would make the macro experience so much nicer to work with. Thanks for the insight in the meantime

[TekExplorer]

Honestly, this and a union-like feature would be great.

Macros are cool, but they still suck when it comes to actually emitting code. You can't tell at a glance what's happening.

If we could get this feature, it would make the lists of objects much easier to parse (for the human) in this way.

Combined with some kind of union feature, weather that's union types, or rust-like traits (ie, implement Foo for Bar) then we could even get proper type safety for it.

(I keep accidentally passing Future into code blocks :p. I wonder if they could straight up accept that naturally, since it's not like the compiler/analyzer can't evaluate futures.)

[Wdestroier]

I keep accidentally passing Future into code blocks.

Could a lint help? The lint could tell you when you're interpolating a Future in a String.

[TekExplorer]

I keep accidentally passing Future into code blocks.

Could a lint help? The lint could tell you when you're interpolating a Future in a String.

Sure, and usually I have it active, but throwing up a new project to test it just doesn't include it automatically for some reason. Plus, I have to imagine that there's no real reason we can't give it a future imo. We're just giving it to an async backend anyway - but that's beside the point. (There's a reason I put it in parens :p)