Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: string interpolation #34174

Open
sirkon opened this issue Sep 8, 2019 · 67 comments
Open

proposal: string interpolation #34174

sirkon opened this issue Sep 8, 2019 · 67 comments

Comments

@sirkon
Copy link

@sirkon sirkon commented Sep 8, 2019

Introduction

For ones who don't know what it is:

Swift

let multiplier = 3
let message = "\(multiplier) times 2.5 is \(Double(multiplier) * 2.5)"
// message is "3 times 2.5 is 7.5"

Kotlin

var age = 21

println("My Age Is: $age")

C#

string name = "Mark";
var date = DateTime.Now;

Console.WriteLine($"Hello, {name}! Today is {date.DayOfWeek}, it's {date:HH:mm} now.");

Reasoning of string interpolation vs old school formatting

I used to think it was a gimmick but it is not in fact. It is actually a way to provide type safety for string formatting. I mean compiler can expand interpolated strings into expressions and perform all kind of type checking needed.

Examples

variable := "var"
res := "123\{variable}321" // res := "123" + variable + "321"
return errors.New("opening config file: \\{err}") // return errors.New("opening config file: " + err.Error())
var status fmt.Stringer
…
msg := "exit status: \{status}" // msg := "exit status: " + status.String()
v := 123
res := "value = \{v}" // res := "value = " + someIntToStringConversionFunc(v)

Syntax proposed

  • Using $ or {} would be more convenient in my opinion, but we can't use them for compatibility reasons
  • Using Swift \(…) notation would be compatible but these \() are a bit too stealthy

I guess {…} and \(…) can be combined into \{…}

So, the interpolation of variable variable into some string may look like

"<prefix>\{variable}<suffix>"

Formatting also has formatting options. It may look like

"<prefix>\{variable[:<options>]}<suffix>"

Examples of options

v := 123.45
fmt.Println("value=\{v:04.3}") // value=0123.450
v := "value"
fmt.Println("value='\{v:a50}'") // value='<45 spaces>value'

etc

Conversions

There should be conversions and formatting support for built in types and for types implementing error and fmt.Stringer. Support for types implementing

type Formatter interface {
    Format(format string) string
}

can be introduced later to deal with interpolation options

Pros and cons over traditional formatting

Pros

  • Type safety
  • Performance (depends on the compiler)
  • Custom formatting options support for user defined types

Cons

  • Complication of a compiler
  • Less formatting methods supported (no %v (?), %T, etc)
@agnivade agnivade changed the title String interpolation may be? proposal: string interpolation Sep 8, 2019
@gopherbot gopherbot added this to the Proposal milestone Sep 8, 2019
@gopherbot gopherbot added the Proposal label Sep 8, 2019
@latitov
Copy link

@latitov latitov commented Sep 9, 2019

Why can't we use just ${...}? Swift already has one syntax, JavaScript and Kotlin another, C# yet another, also Perl... Why invent one more variation? Wouldn't it be better to stick to the one most readable and already existing?

@sirkon
Copy link
Author

@sirkon sirkon commented Sep 9, 2019

Why can't we use just ${...}? Swift already has one syntax, JavaScript and Kotlin another, C# yet another, also Perl... Why invent one more variation? Wouldn't it be better to stick to the one most readable and already existing?

Because we may have existing strings with ${…}. I do, for instance.

And \{ is not allowed right now.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Sep 10, 2019

This doesn't seem to have a big advantage over calling fmt.Sprintf.

@sirkon
Copy link
Author

@sirkon sirkon commented Sep 10, 2019

This doesn't seem to have a big advantage over calling fmt.Sprintf.

Yes, it doesn’t besides full type safety, better performance and ease of use. Formatting done right for a language with static typing.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Sep 10, 2019

As far as I can tell this proposal isn't any more type safe than using fmt.Sprintf with %v. It's essentially identical with regard to type safety. I agree that a careful implementation could have better performance in many cases. I'm agnostic on ease of use.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Sep 10, 2019

In order to implement this we would have to write, in the language spec, the exact formatting to use for all types. We would have to decide and document how to format a slice, an array, a map. An interface. A channel. This would be a significant addition to the language spec.

@latitov
Copy link

@latitov latitov commented Sep 11, 2019

I think it's one of those questions where both ways have the right to exist, and it's just up to the decision makers to decide. In Go, the decision has been already made, quite a long time ago, and it works, and it's idiomatic. That's fmt.Sprintf with %v.

Historically, there are languages where interpolation inside a string has been present from the very beginning. Notably it's Perl. That's one of the reasons why Perl became so popular, because it was a super-convenient compared to sprintf() in C et al. And the %v wasn't invented then yet. And then there are languages where the interpolation was present, kind of, but was inconvenient syntactically, think of "text" + v1 + "text". JavaScript then introduced backtick-quoted literals, which are multi-line, and support interpolation of arbitrary expressions inside ${...}, which was huge improvement compared to "text" + v1 + "text". The Go too has backtick multi-line literals, but without ${...}. Who copied whom I don't know.

I don't agree with @ianlancetaylor that support of this will need substantial effort. In fact, fmt.Sprintf with %v does exactly this, doesn't it? It looks for me like a different syntax wrapper to exactly the same thing under the hood. Am I right?

But I do agree with @ianlancetaylor that using fmt.Sprintf with %v is as convenient. It's also exactly the same in length on the screen, and, what is IMHO very important - is already idiomatic for Go. It kind of makes Go Go. If we copy-implement every other feature from every other language, then it will no longer be Go, but every other language.

There's one more thing. From my long experience with Perl, where string interpolation was present from the very beginning, I can say that it's not that perfect. There's a problem with that. For simple and trivial programs, and probably for 90% of all programs, string interpolation of variables works just fine. But then, once in a while, you get a var=1.99999999999, and want to print it as 1.99, and you can't do that with standard string interpolation. You either need to do some conversion beforehand, or... Look into the docs, and re-learn long forgotten sprintf() syntax. Here is the problem - using string interpolation allows you to forget how to use sprintf()-like syntax, and probably it's very existence. And then when it's needed - you spend too much time and effort to do simplest things. I am talking in Perl context, and it was a decision of language designer(s) to do so.

But in Go, a different decision has been made. The fmt.Sprintf with %v is already here, it's as convenient, as short as interpolated strings, and it's idiomatic, in that it's in the docs, in examples, everywhere. And it doesn't suffer from the problem of eventually forgetting how to print 1.99999999999 as 1.99.

Introducing proposed syntax will make Go a little more like Swift and/or more like JavaScript, and some may like that. But I think this particular syntax will not make it better, if not a little worse.

I think that existing way to print things should stay as it is. And if someone needs more - then there are templates for that.

@mvdan
Copy link
Member

@mvdan mvdan commented Sep 11, 2019

If part of the argument here is safety at compile time, I don't agree that's a compelling argument; go vet, and by extension go test, have been flagging incorrect uses of fmt.Sprintf for a while.

It's also possible to optimize for performance today via go generate, if you really wish to do so. It's a tradeoff that's not worth it most of the time. I feel like the same applies to greatly expanding the spec; the tradeoff is generally not worth it.

@kaey
Copy link

@kaey kaey commented Sep 11, 2019

Allowing function calls inside interpolated strings would be unfortunate - too easy to miss.
Not allowing them is another unnecessary special case.

@conilas
Copy link

@conilas conilas commented Sep 13, 2019

If part of the argument here is safety at compile time, I don't agree that's a compelling argument; go vet, and by extension go test, have been flagging incorrect uses of fmt.Sprintf for a while. It's also possible to optimize for performance today via go generate, if you really wish to do so. It's a tradeoff that's not worth it most of the time. I feel like the same applies to greatly expanding the spec; the tradeoff is generally not worth it.

But type safeness should be guaranteed by the compiler, not by tooling. This is semantics; it is like saying that there should be a tool for verifying where you forgot to null check instead of having optionals or explicitly declared nullable values.

Apart from that - the only way for this to be safe is with dependent types. String interpolation is just more syntatic sugar for the same stuff as fmt.Sprintf and, although I'm all in favor of some good sugar, the whole go community seems not to be.

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Sep 17, 2019

Or perhaps something like modifying the language to just work better with fmt.Printf & friends.

Like if fmt supported something like %(foo)v or %(bar)q, then say that if a string literal containing %(<ident>) is used in a call to a variadic func/method, then all the referenced symbols are appended to the variadic list automatically.

e.g. this code:

name := "foo"
age := 123
fmt.Printf("The gopher %(name)v is %(age)2.1f days old.")

would really compile to:

name := "foo"
age := 123
fmt.Printf("The gopher %(name)v is %(age)2.1f days old.", name, age)

And fmt could just skip over the unnecessary (name) and (age) bits.

That's a pretty special case language change, though.

@alvaroloes
Copy link

@alvaroloes alvaroloes commented Sep 19, 2019

For me, type-checking as a reason to include string interpolation in the language is not that compelling. There is a more important reason, which is not depending on the order in which you write the variables in fmt.Printf.

Let's take one of the examples from the proposal description and write it in Go with and without string interpolation:

  • Without string interpolation (current Go)
name := "Mark"
date := time.Now()

fmt.Printf("Hello, %v! Today is %v, it's %02v:%02v now.", name, date.Weekday(), date.Hour(), date.Minute())
  • Equivalent functionality with string interpolation (and mixing the @bradfitz comment, needed to express the formating options)
name := "Mark"
date := time.Now()

fmt.Printf("Hello, %{name}! Today is %{date.Weekday()}, it's %02{date.Hour()}:%02{date.Minute()} now.")

For me, the second version is easier to read and modify and less error-prone, as we do not depend on the order in which we write the variables.

When reading the first version, you need to re-read the line several times to check it is correct, moving your eyes back and forth to create a mental mapping between the position of a "%v" and the place where you need to put the variable.

I've been fixing bugs in several applications (not written in Go, but with the same issue) where the database queries had been written with a lot of '?' instead of named parameters (SELECT * FROM ? WHERE ? = ? AND ? != false ...) and further modifications (even inadvertently during a git merge) flipped the order of two variables 😩

So, for a language whose goal is to ease maintainability in the long term, I think this reason makes it worth it to consider having string interpolation

Regarding the complexity of this: I don't know the internals of Go compiler, so take my opinion with a grain of salt, but Couldn't it just translate the second version showed in the above example to the first one?

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Sep 25, 2019

@ianlancetaylor pointed out that my sketch above (#34174 (comment)) isn't strictly backwards compatible, as there might be rare programs where this would change their behavior.

A backwards compatible variation would be to add a new type of "formatted string literal", prefixed by, say, an f:

e.g. this code:

name := "foo"
age := 123
fmt.Printf(f"The gopher %(name)v is %(age)2.1f days old.")

would really compile to:

name := "foo"
age := 123
fmt.Printf(f"The gopher %(name)v is %(age)2.1f days old.", name, age)

But then the double f (one in Printf followed by the f before the new type of string literal) would be stuttery.

@alphabettispaghetti
Copy link

@alphabettispaghetti alphabettispaghetti commented Oct 10, 2019

I also don't understand the inner workings of the compiler, so I (perhaps foolishly) also assume that this could be implemented in the compiler, so that something like
fmt.printf("Hello %s{name}. You are %d{age}")

would compile to its equivalent current formulation.

String interpolation has the obvious benefit of higher readability (A core design decision of Go) and also scales better as the strings that one deals with become longer and more complicated (another core design decision of Go). Please notice also that using {age} gives the string context that it otherwise wouldn't have if you were only skim reading the string (and of course ignoring the type that was specified), the string could have ended "You are tall", "You are [at XXX location]", "You are working too hard" and unless you put in the mental energy to map the format method to each instance of interpolation it isn't immediately obvious what should go there. By removing this (admittedly small) mental hurdle the programmer can focus on the logic rather than the code.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Oct 10, 2019

The compiler implements the language spec. The language spec currently says nothing at all about the fmt package. It doesn't have to. You can write large Go programs that don't use the fmt package at all. Adding the fmt package documentation to the language spec would make it noticeably larger, which is another way of saying that it makes the language that much more complex.

That doesn't make this proposal impossible to adopt, but it is a large cost, and we need a large benefit to outweigh that cost.

Or we need a way to discuss string interpolation without involving the fmt package. This is pretty clear for values of string type, or even []byte type, but much less clear for values of other types.

@alanfo
Copy link

@alanfo alanfo commented Oct 11, 2019

I'm not in favor of this proposal partly because of what @ianlancetaylor said above and partly because, when you try to interpolate complex expressions with formatting options, any readability advantage tends to go out of the window.

Also it's sometimes forgotten that the ability to include variadic arguments in the fmt.Print (and Println) family of functions already enables a form of interpolation. We can easily reproduce some of the examples quoted earlier with the following code which, to my mind, is just as readable:

multiplier := 3
message := fmt.Sprint(multiplier, " times 2.5 is ", float64(multiplier) * 2.5)

age := 21
fmt.Println("My age is:", age)

name := "Mark"
date := time.Now()
fmt.Print("Hello, ", name, "! Today is ", date.Weekday(), ", it's ", date.String()[11:16], " now.\n")

name = "foo"
days := 12.312
fmt.Print("The gopher ", name, " is ", fmt.Sprintf("%2.1f", days), " days old\n.")
@latitov
Copy link

@latitov latitov commented Oct 16, 2019

Another reason to still add and have it in the language: #34403 (comment)

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Oct 29, 2019

We find @alanfo 's comments in #34174 (comment) to be convincing: you can use fmt.Sprint to do a simple sort of string interpolation. The syntax is perhaps less convenient, but any approach at this would require a special marker for variables to be interpolated in any case. And, as noted, this permits arbitrary formatting of the values to be interpolated.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Nov 26, 2019

As noted above there is an existing way to approximately do this that even allows for formatting of the individual variables. Therefore, this is a likely decline. Leaving open for four weeks for final comments.

@runeimp
Copy link

@runeimp runeimp commented Dec 21, 2019

I'm regularly faced with building text blocks with well over 50 variables to insert. Over 70 in a few cases. This would be easy to maintain with Python's f-strings (similar to C# mentioned above). But I'm handling this in Go instead of Python for several reasons. The initial setup of fmt.Sprintf to manage these blocks is... ok. But god forbid I have to fix a mistake or modify the text in any way at all that involves moving or deleting %anything markers and their position related variables. And manually building maps to pass to template or setting up os.Expand is not a great option either. I'll take the speed (of setup) and ease of maintainability of f-strings over fmt.Sprintf any day of the week. And no, fmt.Sprint would not be hugely beneficial. Easier to setup than fmt.Sprintf in this case. But it loses much of it's meaning visually because your jumping in and out of strings. "My {age} is not {quality} in this discussion" doesn't jump in and out of strings the way that "My ", age, " is not ", quality, " in this discussion" does. Especially over the course of many tens of references. Moving text and references around is just copy and paste with f-strings. Deletions are just select and delete. Because your always within the string. This is not the case when using fmt.Sprint. It's very easy to accidentally (or necessarily) select non-string commas or double-quote string terminations and move them about breaking the formatting and requiring edits to massage it back into place. fmt.Sprint and fmt.Sprintf in these cases is far more time consuming and error prone than anything resembling f-strings.

@marcstreeter
Copy link

@marcstreeter marcstreeter commented Jan 18, 2020

Could it be considered to leave this discussion open for one more set of 4 weeks, since a sizable portion of the community was on vacation for Thanksgiving, Christmas and New Years when the cutoff was proposed?

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jan 21, 2020

There has been a bunch more discussion since #34174 (comment), so I'm taking this back out of final-comment-period.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jan 21, 2020

I am inclined to agree with @randall77 that I have not yet seen a compelling example here. @runeimp, thanks for posting the example code, but it seems difficult to read and difficult to change either way. As @egonelbre suggests, if we want to make this more maintainable, the first step seems to be to find an entirely different approach.

@cyclingwithelephants fmt.sprintf("I am %T{age} years old") is not on the table here. The language does not provide any mechanism that fmt.Sprintf could use to resolve age. Go is a compiled language, and local variable names are not available at execution time. This would be more palatable if we could figure out some way to make that work, perhaps along the lines of @bradfitz's suggestion above.

@runeimp
Copy link

@runeimp runeimp commented Jan 21, 2020

Thank you @ianlancetaylor for lifting the final-comment-period label. 😀

I think @bradfitz's idea was great. I'd guess there are potential limitations no matter what without a locals variable context but I'd happily accept those limitations over not having string interpolation. While I have mad respect for the updates in percentage formatting in Go (I love the additions of %q, %v, and %#v) that paradigm is ancient. Just because it's venerable does not mean is the best way to do things. Just like Go's way of handling compiling, and especially cross-compiling is WAY better than how it's done with C or C++. Now, does Go just hide all the nasty compiler options with sane defaults and it's just as ugly under-the-hood. I don't know specifically but I believe that is the case. And that is fine. That is completely acceptable to me. I don't care what dark ritual the compiler does to make the feature work. I just know the feature makes life easier. And has made my life easier as a developer in every language that I've used that supports it. And is always a pain in languages that don't support it. It's significantly easier to remember than how to use the 30+ special characters in percent formatting for 98% of the string formatting I need. And saying to use %v instead of "the correct percentage format" is not the same ease of use for creation nor even close the same ease of maintenance.

Now that there is a bit more time I'll work on an example and see if I can find some articles that are little more enlightening than I have been to help illustrate the significant benefits to work efficiency for those of us who deal in human interfacing, document and code generation, and string manipulation on a regular basis.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jan 22, 2020

Here is a thought that may lead to something implementable. Though I don't know that it is a good idea.

Add a new string type m"str" (and perhaps the same with a raw string). This new kind of string literal evaluates to a map[string]interface{}. Looking up the empty string in the map gives you the string literal itself. The string literal may contain expressions in braces. An expression in braces is evaluated as though it were not in the string literal, and the value is stored in the map with the key being the substring that appears within the braces.

For example:

    i := 1
    m := m"twice i is {i * 2}"
    fmt.Println(m[""])
    fmt.Println(m["i * 2"])

This will print

twice i is {i * 2}
2

Within the string literal, braces may be escaped with a backslash to indicate a simple brace. An unquoted, unmatched, brace is a compilation error. It is also a compilation error if the expression within the braces cannot be compiled. The expression must evaluate to exactly one value but is otherwise unrestricted. The same braced string may appear multiple time in the string literal; it will be evaluated as many times as it appears, but only one of the evaluations will be stored in the map (because they will all have the same key). Exactly which one is stored is unspecified (this matters if the expression is a function call).

By itself this mechanism is peculiar yet useless. Its advantage is that it can be clearly specified and arguably does not require excessive additions to the language.

The use comes with additional functions. The new function fmt.Printfm will work exactly like fmt.Printf, but the first argument will not be a string but rather a map[string]interface{}. The "" value in the map will be a format string. The format string will support, besides the usual % things, a new {str} modifier. The use of this modifier will mean that instead of using an argument for the value, str will be looked up in the map, and that value will be used.

For example:

    hi := "hi"
    fmt.Printfm(m"%20{hi}s")

will print the string hi passed out to 20 spaces.

Naturally there will be the simpler fmt.Printm which will substitute for each braced expression the contained value as printed by fmt.Print.

For example:

    i, j := 1, 2
    fmt.Printm(m"i: {i}; j: {j}")

will print

i: 1; j: 2

Problems with this approach: the odd use of an m prefix before a string literal; the duplicated m in normal use--one before and one after the parenthesis; the general uselessness of an m-string when not used with a function that expects one.

Advantages: not too hard to specify; supports both simple and formatted interpolation; not limited to fmt functions, so may work with templates or unanticipated uses.

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Jan 22, 2020

If it were a typed map (e.g. runtime.StringMap) then we could use fmt.Print and Println without adding the stuttery Printfm

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jan 22, 2020

Using a defined type is a good idea, but it wouldn't help with fmt.Printfm; we couldn't use the defined type as the first argument to fmt.Printf, since that only takes a string.

@alanfo
Copy link

@alanfo alanfo commented Jan 22, 2020

One thing which was mentioned earlier in the thread by @runeimp but hasn't been fully discussed is os.Expand:-

package main

import (
    "fmt"
    "os"
)

func main() {
    name := "foo"
    days := 12.312
    type m = map[string]string
    f := func(ph string) string {
        return m{"name": name, "days": fmt.Sprintf("%2.1f", days)}[ph]
    }
    fmt.Println(os.Expand("The gopher ${name} is ${days} days old.", f))
    // The gopher foo is 12.3 days old.
}

Although this is too verbose for simple cases, it's much more palatable when you have a largish number of values to be interpolated (though any approach has a problem with 70 values!). Advantages include :-

  1. If you use a closure for the mapping function, it deals fine with local variables.

  2. It also deals fine with arbitrary formatting and keeps that out of the interpolated string itself.

  3. If you use a placeholder in the interpolated string which isn't present in the mapping function it automatically gets replaced with an empty string.

  4. Changes to the mapping function are relatively easy to make.

  5. We already have it - no languge or library changes are necessary.

@runeimp
Copy link

@runeimp runeimp commented Jan 22, 2020

@ianlancetaylor that solution sounds like a solid option to me. Though I'm not seeing why we need alternate Print methods. I'm likely overlooking something but seems a simple signature change using interface{} and type check. OK, just realized how the signature change could prove very problematic for some existing code. But if the basic mechanism was implemented and and we also created a stringlit type that represents string or m"str" or if m"str" also accepted string would that be an acceptable breaking change for Go v2? They are both "string literals", it's just that one of them essentially has a flag that allows for extra functionality, no?

@runeimp
Copy link

@runeimp runeimp commented Jan 22, 2020

Thanks for bringing that up again @alanfo, those are all excellent points. 😃

I've used os.Expand for light templating and it can be very handy in situations where you need to build a map of values anyway. But if the map isn't needed, and you would need to make your closure in several different areas just to capture the local variables for your (now copied many times) replacement function ignores DRY entirely and can will lead to maintenance issues and just adds more work were interpolated strings would "just work", alleviate those maintenance issues, and not require the building of a map just to manage that dynamic string.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jan 22, 2020

@runeimp We can't change the signature of fmt.Printf. That would break Go 1 compatibility.

The notion of a stringlit type implies changing Go's type system, which is a much bigger deal. Go intentionally has a very simple type system. I don't think we want to complicate it for this feature. And even if we did, fmt.Printf would still take a string argument, and we can't change that without breaking existing programs.

@runeimp
Copy link

@runeimp runeimp commented Jan 22, 2020

@ianlancetaylor Thanks for the clarification. I appreciate the desire to not break backwards compatibility with something as fundamental as the fmt package or the type system. I was just hoping there might be some hidden (to me) possibility that might be an option along those lines somehow. 👼

@alvaroloes
Copy link

@alvaroloes alvaroloes commented Jan 23, 2020

I really like the Ian way to implement this. Wouldn't generics help with the fmt.Print issue?

contract printable(T) {
  T string, map[string]string // or the type Brad suggested "runtime.StringMap"
}

// And then change the signature of fmt.Print to:
func Print(type T printable) (str T) error { 
  // ...
}

This way, the Go 1 compatibility should be preserved.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jan 23, 2020

For Go 1 compatibility, we can't change the type of a function at all. Functions are not only called. They are also used in code like

    var print func(...interface{}) = fmt.Print

People write code like this when making tables of functions, or when using hand-rolled dependency injection for tests.

@sbourlon
Copy link

@sbourlon sbourlon commented Feb 2, 2020

I have the feeling that strings.Replacer (https://golang.org/pkg/strings/#Replacer) can almost do string interpolation, just missing the interpolation identifier (e.g. ${...}) and the pattern processing (e.g. if var i int = 2, "${i+1}" should be mapped to "3" in the replacer)

@beoran
Copy link

@beoran beoran commented Feb 5, 2020

Yet another approach would have a built-in function, say, format("I am a %(foo)s %(bar)d") that expands to fmt.Sprintf("I am a %s %d", foo, bar). At least, that's fully backwards compatible, FWIW.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Feb 5, 2020

From a language design perspective, it would be peculiar to have a builtin function expand to a reference to a function in the standard library. To provide a clear definition for all implementations of the language, the language spec would have to fully define the behavior of fmt.Sprintf. Which I think we want to avoid.

@jimmyfrasche
Copy link
Member

@jimmyfrasche jimmyfrasche commented Feb 21, 2020

This probably won't make everyone happy but I think the below would be the most general. It's broken up into three parts

  1. fmt.Printm functions that take a format string and a map[string]interface{}
  2. accept #12854 so you can drop the map[string]interface{} when calling it
  3. allow unkeyed names in map literals as shorthand for "name": name, or "qual.name": qual.name,

Taken together that would allow something like

fmt.Printm("i: {i}; j: {j}", {i, j})
// which is equivalent to
fmt.Printm("i: {i}; j: {j}", map[string]interface{}{
  "i": i,
  "j": j,
})

That still has the duplication between the format string and the arguments but it's a lot lighter on the page and it's a pattern that's easily automated: an editor or tool could automatically fill in the {i, j} based on the string and the compiler would let you know if they're not in scope.

That doesn't let you do computations within the format string which can be nice, but I've seen that overdone enough times to consider it a bonus.

Since it applies to map literals in general it can be used in other cases. I often name my variables after the key they'll be in the map I'm building.

A downside of this is that it can't apply to structs since those can be unkeyed. That could be rectified by requiring a : before the name like {:i, :j} and then you could do

Field2 := f()
return aStruct{
  Field1: 2,
  :Field2,
}
@beoran
Copy link

@beoran beoran commented Feb 24, 2020

Do we need any language support for this? As go is now, it can look like this, either with a map type or with a fluid, more type-safe API:

package main

import (
	"fmt"
	"strings"
)

type V map[string]interface{}

func Printm(format string, args V) {
	for k, v := range args {
		format = strings.ReplaceAll(format, fmt.Sprintf("{%s}", k), fmt.Sprintf("%v", v))
	}
	fmt.Print(format)
}

type Buf struct {
	sb strings.Builder
}

func Fmt(msg string) *Buf {
	res := Buf{}
	res.sb.WriteString(msg)
	return &res
}

func (b *Buf) I(val int) *Buf {
	b.sb.WriteString(fmt.Sprintf("%v", val))
	return b
}

func (b *Buf) F(val float64) *Buf {
	b.sb.WriteString(fmt.Sprintf("%v", val))
	return b
}

func (b *Buf) S(val string) *Buf {
	b.sb.WriteString(fmt.Sprintf("%v", val))
	return b
}

func (b *Buf) Print() {
	fmt.Print(b.sb.String())
}

func main() {
	Printm("Hello {k} {i}\n", V{"k": 22.5, "i": "world"})
	Fmt("Hello ").F(22.5).S(" world").Print()
}

https://play.golang.org/p/v9mg5_Wf-qD

Ok, it's still inefficient, but it looks like it is not a lot of work at all to make a package that supports this. As a bonus, I included a different fluid API that might be said to simulate interpolations somewhat as well.

@HALtheWise
Copy link

@HALtheWise HALtheWise commented Apr 28, 2020

The "map string" proposal from @ianlancetaylor (although I personally prefer "value/variable string" with a v"..." syntax) also allows non-formatting use cases. For example, #27605 (operator overloading functions) largely exists because it is difficult today to make a readable API for math/big and other numeric libraries. This proposal would allow the function

func MakeInt(expression map[string]interface{}) Int {...}

Used as

a := 5
b := big.MakeInt(m"100000")
c := big.MakeInt(m"{a} * ({b}^2)")

Importantly, this helper function can coexist with the more performant and powerful API that currently exists.

This approach allows the library to perform whatever optimizations it wants for large expressions, and might also be a useful pattern for other DSLs, since it allows custom expression parsing while still representing values as Go variables. Notably, these use cases are not supported by Python's f-strings because the interpretation of the enclosed values is imposed by the language itself.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Apr 30, 2020

@HALtheWise Thanks, that is pretty neat.

@slycrel
Copy link

@slycrel slycrel commented May 18, 2020

I wanted to comment to show a little support for this proposal, from the stance of a general developer. I've been coding with golang for over 3 years professionally. When I moved to golang (from obj-c/swift) I was disappointed that string interpolation was not included. I've used C and C++ for over a decade in the past, so printf wan't a particular adjustment, other than feeling like going backwards a little -- I've found that it does indeed make a difference with code maintenance and readability for more complex strings. I've recently done a little bit of kotlin (for a gradle build system), and using string interpolation was a breath of fresh air.

I think string interpolation can make string composition more approachable for those new to the language. It's also a win for technical UX and maintenance, due to the reduction of cognitive load both when reading and writing code.

I am glad that this proposal is getting real consideration. I look forward to the resolution of the proposal. =)

@rodcorsi
Copy link

@rodcorsi rodcorsi commented May 18, 2020

If I understand correctly, the @ianlancetaylor proposal is:

i := 3
foo := m"twice i is %20{i * 2}s :)"
// the compiler will expand to:
foo := map[string]interface{}{
	"": "twice i is %20{i * 2}s :)",
	"i * 2": 6,
}

After that, a print function will handle that map, parse entire template again, and take few advantages of the pre-parsed template

But if we expand m"str" to a function?

i := 3
foo := m"twice i is %20{i * 2}s :)"
// the compiler will expands to:
foo := m(
	[]string{"twice i is ", " :)"}, // split string
	[]string{"%20s"},               // formatter for each value
	[]interface{}{6},               // values
)

This function has the following signature:

func m(strings []string, formatters []string, values []interface{}) string {}

This function will perform better because, to take more advantage of the pre-parsed template, and much more optimizations could be done similar as Rust does with the println! function.

What I'm trying to describe here is very similar to the Tagged functions of the Javascript, and we could discuss whether the compiler should accept user functions to format string ex:

foo.GQL"query { users{ %{expectedFields} } }"

bla.SQL`SELECT *
	FROM ...
	WHERE FOO=%{valueToSanitize}`
@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented May 18, 2020

@rodcorsi If I'm reading your suggestion correctly, it requires building fmt.Printf formatting into the language proper, because the compiler will have to understand where %20s starts and ends. That is one of the things I was trying to avoid.

Also note that my suggestion is not at all tied to fmt.Printf formatting, and can be used for other kinds of interpolation as well.

@HALtheWise
Copy link

@HALtheWise HALtheWise commented May 18, 2020

I would be opposed to treating m"..." as expanding to a function call, because it obscures what's actually going on, and adds what's effectively a second syntax for function calls. It does generally seem reasonable to pass a more structured representation than a map, to avoid needing matching reimplementations of the parsing behavior everywhere. Perhaps a simple struct with a slice of constant string sections, a slice of strings for things in braces, and an interface slice?

m"Hello {name}" -> 
struct{...}{
    []string{"Hello ", ""},
    []string{"name"},
    []interface{}{"Gopher"}

The second and third slices must be the same length, and the first must be one longer. There are other ways to represent this as well to encode that constraint structurally.
The advantage this has over a format that directly exposes the original string is that there's a looser requirement to have a correct and performant parser in the function that consumes it. If there's no support for escaped characters or nested m-strings, it probably isn't a big deal, but I'd rather not need to reimplement and test that parser, and caching it's result could cause runtime memory leaks.

@HALtheWise
Copy link

@HALtheWise HALtheWise commented May 18, 2020

If "formatting options" is a frequent desire of things using this syntax, I could see there being a place for them in the spec, but I'd personally go with a syntax like m"{name} is {age:%.2f} years old" where the compiler just passes everything after the : on to the function.

@Frietziek
Copy link

@Frietziek Frietziek commented May 22, 2020

Hello I wanted to comment on this to add support to this proposal. I been working with many different languages in the past 5 years (Kotlin, Scala, Java, Javascript, Python, Bash, some C, etc) and I'm learning Go now.

I think string interpolation is a must have in any modern programming language, the same way as type inference is, and we have that in Go.

For those arguing that you can accomplish the same thing with Sprintf, then, I don't understand why we have type inference in Go, you could accomplish the same thing writing the type right? Well, yes, but the point here is that string interpolation reduce a lot of the verbosity you need to accomplish that and is more easy to read (with Sprintf you have to jump to the argument list and the string back and forth to make sense of the string).

In real life software, this is a much appreciate feature.

It is against the Go minimalist design? No, it's not a feature that allow you to do crazy things or abstractions that complicate your code (like inheritance), is just a way of writing less and add clarity when you read the code, which I believe isn't against what Go is trying to do (we have type inference, we have the := operator, etc).

@Skalnark
Copy link

@Skalnark Skalnark commented Aug 17, 2020

Formatting done right for a language with static typing

Haskell has libraries and language extensions for string interpolation. It's not a type thing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.