[RFC FS-1001 Discussion] String Interpolation

[v2m]

This issue is used to track discussions of F# RFC FS-1001 - "String interpolation"

[dsyme]

I presume the approach of using Concat is preferred over generating "%d%O%d%O" because of the difficult in inserting "foo" and "bar.bar" at the right points in the argument list?

[dsyme]

Re "Should embedded expressions be restricted to just identifiers\dotted names or we should allow full set of F# expressions" -

I'm fairly strongly inclined to the position that allowing arbitrary expressions will hurt code readability, somewhat needlessly. Restricting to identifiers or a very simple class of expressions seems sensible.

[dsyme]

Re: "Should we provide ways to specify width\precision\alignment similar to what [printf][6] is doing today? If yes - what modification should be made to the proposed syntax?"

I don't tend to like language features that lead to a non-localized discontinuity when you want to "do the obvious next thing" in a programming sequence - for example, format a displated number 5.6234 to two decimal places - 5.62. These kinds of discontinuities are a PITA for teachers - do they teach the "simple but ultimately way too limited" version of printing - %(foo), or do they teach the "complete but more complex" way of doing things - %d, %0.2d etc.

So my inclination is that we need to handle width/precision/alignment specifiers. Does %0.2(foo) seem reasonable?

[v2m]

I presume the approach of using Concat is preferred over generating "%d%O%d%O" because of the difficult in inserting "foo" and "bar.bar" at the right points in the argument list?

Yes, this is one of reasons. i.e. when embedded expressions in format string are interleaved with format specifiers then printf "%d %(foo) %s" should be transformed to fun a b -> printf "%d %s %s" a foo s. Given that user has already specified what values should be put in a placeholders of the format string I see no point in adding extra complexity to implementation (creating an extra closure) and to runtime (avoid manufacturing the function in runtime for cases when format string uses only embedded expressions)

I'm fairly strongly inclined to the position that allowing arbitrary expressions will hurt code readability, somewhat needlessly. Restricting to identifiers or a very simple class of expressions seems sensible.

This is an interesting question from the implementation perspective. We already have implementation for the one extreme: allowing arbitrary F# expression - in this case we can just reuse existing parser. Another extreme: allowing just identifiers\dotted expressions - writing parser for this case will be trivial. Any other options will require making customized parser that will recognize only limited subset of expressions. Personally I would not go in this direction and allow anything so it will be responsibility of the consumer to write the readable code.

Does %0.2(foo) seem reasonable?

Yes, I like it.

[forki]

Unfortunately the corresponding pull request on codeplex was closed because of release pressure. Can we somehow revive this? String interpolation is a feature that a lot of users would like to see.

[forki]

Ok I rebased it on master: dotnet/fsharp#921

[enricosada]

a question: are we supporting localization correctly?

msdn about c# string interpolation

It's not only about a better string format, need to support i18n too. C# does that because string interpolation does implicit type conversion to FormattableString

var s = $"hello, {name}"
System.IFormattable s = $"Hello, {name}"
System.FormattableString s = $"Hello, {name}"

so

var product = "Raspberry Pi";
var price = 25.0;
Console.WriteLine(LocalizationHelper.Format(
    $"The price of {product} is {price}.";
));

However, the string interpolation will be compiled to a FormattableString with “The price of {0} is {1:C}” as the format that will get passed to GetLocalized. In other words, the people doing the localization will still have to localize with numeric indices instead of the more readable named tokens

If the preferred way for f# is explitic, i think we need another function like $ to formattablestring

[jarlestabell]

(I posted this originally in the wrong thread (the implementation thread).)

I assume most people here know the Scala solution to this, but just in case I want to point that one out, as I think it is quite elegant.
I assume it is drastically much heavier to implement though and it may overlap a bit with type provider territory.

The key point is that it has a concise syntax and is extensible.
So the same concept can be used both for string interpolation without formatting:

val name = "James"
println(s"Hello, $name")  // Hello, James

with formatting:

val height = 1.9d
val name = "James"
println(f"$name%s is $height%2.2f meters tall")  // James is 1.90 meters tall

The above examples are taken from:
http://docs.scala-lang.org/overviews/core/string-interpolation.html

And plain SQL queries: (http://slick.typesafe.com/doc/3.1.1/sql.html#string-interpolation)

def insert(c: Coffee): DBIO[Int] =
 sqlu"insert into coffees values (${c.name}, ${c.supID}, ${c.price}, ${c.sales}, ${c.total})"

and even things like ReactJS-like JSX/TSX xml-based syntax:

xml”””
      <body>
        <a href = “some link”> ${linktext} </a>
    </body>
    ”””

(example from http://docs.scala-lang.org/sips/completed/string-interpolation.html)

[enricosada]

scala syntax and extensibility it's really interesting, StringContext it's like System.FormattableString and the rest it's like sprintf.
The extensibility is really nice to have to make a FormattableString version (useful for localization)

[jarlestabell]

Note that ES6 seems to have the same syntax as Scala, even down to the extensibility. ("tagged template literals")
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals
When working in F#, Scala's string interpolation is one of the few things I miss from Scala.
(Although I have done type mistakes with doing string interpolation in Scala)

[cloudRoutine]

I'm with @isaacabraham on using the "%{expr}" syntax over parentheses. Especially if they allow arbitrary expressions, then it is syntactically coherent with the F# conventions of using % for substitution and { } to delineate a block of scope.

Since we'll know the type the expression resolves to we can determine which formatting specifications to enable. For most types -

• %10{**expr**} - sets the width of the embedded expression
• %*{**expr**} - sets the width dynamically
• %-{**expr**} - left justifies the embedded expression

If it resolves to a numeric type

• %0{**numeric-expr**} - pads with zeros
• %+{**numeric-expr**} - shows a plus sign
• % {**numeric-expr**} - shows a blank in place of a plus sign
• %.5{**numeric-expr**} - control precision when expr is a float

The primitive types can follow the same style of formatting that printf uses, defaulting to

%b, %c, %i, %u, %x, %o, %f, %M,

Additional Questions -

• Is interpolation allowed in every string, only printf strings or will it need a prefix operator?
• Should interpolation be allowed in bare strings if only interpolation substitution is used?
  e.g. "hello %{name}, I have your %10{item}"
• Would this be valid "%{root}"B ?
• Should there be an additional specifier to indicate %O format instead of %A format, or vice versa depending on whichever becomes the default.

[smoothdeveloper]

@cloudRoutine sorry for being late in the party, but you say

and { } to delineate a block of scope.

Would it be conflicting to use ( ) instead? it seems what we want to interpolate with is F# expressions, and parens are used to delineate those more often than curly braces which are only used for records, I think this would be more consistent and would look less strange if one decides to interpolate with an inline record expression or object expression (having %{{field1=1}} would look kind of strange).

[cloudRoutine]

@smoothdeveloper { } are used for record expressions, in constructor overloads, object expressions, seq expressions, query expressions, and computation expressions. In each of these cases the curly braces demarcate a context with slightly different semantics than the scope that encompasses it - through context specific keywords, field assignments, member and interface definitions, or custom operations.

( ) are used to indicate the order in which expressions are evaluated, to delimit a tuple, to use an active pattern or operator like a function, or to represent unit. Whereas { } are always used to define a distinct scope.

" %{{field1=1}}" - declaring an anonymous record expression in the middle of a string interpolation seems like a very unlikely use case.

" %((20.M, 10.0, 5))" - a tuple substitution seems just as if not more probable than your "strange" case if parens were used as the delimiters.

And if for some bizarre reason you feel the need to define a bunch of expressions within the scope of the substitution block, with several lambdas and function calls, there'll be parens all over the place. But it'll be nice and easy to see the bounds around it, thanks to those { } 😜

[smoothdeveloper]

@cloudRoutine thanks, it makes more sense with the expanded description, I took scope as something akin to how it is used in C# but not as being different context as you explain.

As for what goes in the interpolated strings, you'd be surprised of what can be found in code in the wild even though that feature was only recently added to C# 6, I won't be surprised to find tuples or records, and why not object expression since it can be done :)

[cloudRoutine]

By adding another printf format specifier to use with interpolateds string we could add an even stronger degree of type safety.

instead of needing to write helper functions like

 let makeString (arg:MyDu) =
    sprintf "adjsk -- %O" arg

the %T specifier could be used with an interpolated insertion that only accepts types

sprintf "adjsk -- %T{MyDu}" MyDu.Case1

which may not immediately seem useful, but for partial application it could provide a lot of value.

perhaps %T could use StructuredFormatDisplay like %A and a different specifier would use ToString like %O? (or vice versa)

[vasily-kirichenko]

I think we are in a classical "analysis paralysis" :) What about implementing a minimal functionality first, keeping the door open for further improvements?

• choose the C# syntax as is: $"{ }
• no Scala-like extensions
• no specifiers at all
• F# types are formatted with %+A, everything else - with %O

So we would ended up with this:

let s = $"This feature is {x}!" "awesome"

Great things about it:

• no need to switch your mind between two syntaxes when working on C# - F# solutions
• no need for sprintf, which looks old fashioned compared to C# and other langs implementations
• specifiers and extensions can be added later, no breaking changes
• quite easy to implement?

In short, I believe we should implement anything to move forward with this, then, according to feedback, make it great.

[jhurdlow]

I'm with vasily here... Go with the existing C# syntax. It's clean and it works. Most importantly it will be a natural transition for those of us who have to bounce back and forth between C# and F# all day. I find myself trying to do this all the time in F#, then remembering it's (sadly) not there (yet). If you need custom formatting, just do it inside the {}, after all it's just an expression. This is the feature I miss most from C# when working in F#, so it would be nice to see this feature sooner rather than later.

[realvictorprm]

I really would like to have this feature soon in F# with the same syntax it's implemented in C# because I also think it works very well!

Is there anything we can do to continue the discussion and or to receive an approval?

[dsyme]

@realvictorprm Yes, we need to find a way to progress this. We need to go back to some basics - e.g. I'm now inclined to agree we should go with the C# syntax for this

[realvictorprm]

Hm as soon as it's approved I would love to try implementing it to get a grasp of how adding features to the compiler works.

[dsyme]

@realvictorprm Adding string interpolation is "approved in principle" (i.e. "planned" in https://fslang.uservoice.com/forums/245727-f-language/suggestions/6002107-steal-nice-println-syntax-from-swift)

Next step is either to implement the C# syntax, or edit the RFC to capture that as a proposed design

[realvictorprm]

So I could just start looking how to start implementing it, right?

I've a pretty big interest in this because generator stuff etc. involves big string interpolation. On the C# side the code is very readable thanks to the interpolation feature but with F# readability is lacking here really much.

Is there any pointer you can give me where to code should start? E.g the place where strings are being checked?

[vasily-kirichenko]

@realvictorprm see initial attempt dotnet/fsharp#921

[mrange]

And a new case:
pars.fsy, L2657: | STRING_INTERPOLATED { SynConst.StringInterpolated $1 }

StringInterpolated is your new AST case.

[realvictorprm]

yeah I'm a bit before this point, I just added all necessary stuff in the parser and lexer. Now after I understand how it works it's a lot easier 😅

[realvictorprm]

Thank you very much for your advice @mrange. I managed to get the information into the type checker.

The new PR for C# like String-Interpolation is located here!

[dsyme]

@realvictorprm Great work!

[dmitry-a-morozov]

I cannot find here what's the final decision on FormattableString implicit conversion but I would like to point out to this api
https://docs.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.relationalqueryableextensions.fromsql?view=efcore-2.0#Microsoft_EntityFrameworkCore_RelationalQueryableExtensions_FromSql__1_System_Linq_IQueryable___0__System_FormattableString_
And related discussion:
dotnet/efcore#9734 (comment)
dotnet/efcore#10080.

Not defending EF Core team decision but reality is more APIs like that will show up and it would be nice to have smooth inter-op in F#.

[realvictorprm]

I didn't saw that one @dmitry-a-morozov.

I just went through it a small bit and it sounds like a formatting function or whatever. Just something which wraps the operation which would be done instantly. However as always I highly discourage any and I really mean any implicit conversion in F#. Therefore if we want interop with other stuff out there we must acquire it with either 1. a different syntax or 2. we change our mind about interpolated strings in such a way that they are functions which can be executed (which is very unlikely).

I'm pretty open for using another prefix for strings to signal that the formatting shouldn't be executed instantly rather instead a function or formatting object should be returned (maybe it's a function string then 😛 ?). This however would be the same as shortcutting the sprintf function @dmitry-a-morozov I think. Maybe something like this (prefix stands for function)?

let formatDefinition = f"let {name} = {definition}"

[iddan]

What is the status of this?

[cartermp]

@iddan There have been a few PRs from the community, but none have been merged.

[iddan]

What blocks them from being merged?

[cartermp]

We hold a high bar for features getting merged into the compiler, and that means a significant time investment on behalf of us and the person implementing it. We haven't prioritized the feature above other things to do ourselves, so that time investment lies on a community member if they have a strong desire to see the feature in an upcoming version of the language.

[iddan]

Is there a transparent lifecycle of a proposal?
For example: ECMAScript Proposals, Rust rfcbot

[enricosada]

There are info about process in the README

More info this specific issue status in the RFC doc https://github.com/fsharp/fslang-design/blob/master/RFCs/FS-1001-StringInterpolation.md

[yatli]

according to dotnet/fsharp#3593, there are currently two work items:

1. test cases
2. recursive parsing on nested interpolation

2) shall be also covered in 1). So effectively there's one single thing to do -- some TDD challenge/response iterations :)

I suggest, if we care enough about this feature, we can at least "crowdsource" some test cases.

[yatli]

I'm trying to bring it back upon current master.

TODO:
make string interpolation work with ServiceLexing. Looks like the service wasn't there back then.

TODO:
a lot of behaviors/comments are duplicated from bare strings. should be updated.

TODO:
@realvictorprm addressing your question on recursive parsing:
I think the interpolation should be integrated deeper with lex.fsl so that it emits tokens inside interpolated fragments. In this way we can get rid of recursive parsing problems altogether by alleviating the stuff to yacc. Adding a lex state STRING_INTERPOLATING and some minimal housekeeping e.g. STRING_INTERPOLATION_DEPTH would do.
Then the fragments can be marked parsed as STRING_INTERPOLATION_LEFT, STRING_INTERPOLATION_EXPR, STRING_INTERPOLATION_RIGHT whereas STRING_INTERPOLATION_EXPR just involves standard expression parsing. really no need to ad-hoc get a parser when you later see it.

TODO:
interpolated strings should generally not be constants, right? If so during typecheck, it should be unified with a call to formatter not just a constant.

[yatli]

creating a temp PR here: https://github.com/yatli/visualfsharp/pull/1

edit:

compiler's up. feel free to suggest test patterns. I'll work on the TODOs

[yatli]

I'd suggest decompose FS-1001 into two components, and make them more native to what we already have by using simple transformations of the syntax tree rather than introducing new mechanisms:

Part 1: transform $"some string" to (sprintf "some string")
Part 2: transform "some %{interpolated} string" to "some %A string" interpolated -- the formatting options can also be given inline, for example, transform "some %{s:interpolated} string" should be transformed to: "some %s string" interpolated.

In this way string interpolation will work well with the printf family.

I'll have to think about mixing inline arguments with plain formatting placeholders -- the order is the key, for example: $"this is the %d-th %{s:interpolated} string" nr_str -- when transforming the AST we have to take the poisition of nr_str into consideration, or make it two-step formatting (which brings some overhead though)

In this way we don’t even need a new ast case. I’ll ping back when I make progress.

[yatli]

Actually part 1 can be done in the library not the compiler, by making $ an operator that works just like sprintf.

[yatli]

After some thoughts, i think it’s impossible to have one-level formatting if inline arg and placeholders are mixed. The correct way of handling this is to create a closure that captures the inline argument, and output a new formatter.

[yatli]

I'm opening a draft PR, and I will begin documenting my design.

[Alex2357]

Completely agree with @vasily-kirichenko

I think we are in a classical "analysis paralysis" :) What about implementing a minimal functionality first, keeping the door open for further improvements?

If this will work

var s = $"hello, {name}"

it would cover 85-95% of my problems. For the remaining ones I can write sprintf. I believe I'm not alone and most of the people would be quite happy just having this.

[realvictorprm]

It's not like I've done that in the past lol. Even if I started it only :D

[dsyme]

After far too long I've added a comment about the design for this RFC here: dotnet/fsharp#6770 (comment)

[realvictorprm]

Thanks :)! Don Syme <notifications@github.com> schrieb am Mo., 1. Juli 2019, 15:40:

…

After far too long I've added a comment about the design for this RFC here: dotnet/fsharp#6770 (comment) <dotnet/fsharp#6770 (comment)> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#6?email_source=notifications&email_token=AGB4Z4HVA67UL2LHI5I4U3DP5ICNHA5CNFSM4ARLNFR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODY6E7JI#issuecomment-507269029>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGB4Z4GYOBBFQ2RN5AZHUZ3P5ICNHANCNFSM4ARLNFRQ> .

[dsyme]

I've rewritten the RFC. Closing in favour of new discussion thread #368