proposal: spec: allow explicit conversion from function to 1-method interface

[Merovius]

This is forked from #21670. I'd be fine having this closed as a duplicate, but some people suggested that it was different enough to warrant its own tracking issue. As the title suggests, I would like to be able to convert function values to single-method interfaces, if their respective signatures match.

Motivation

My motivation is the same as that proposal. I don't want to have to repeatedly create a type of the form

type FooerFunc func(A, B) (C, D)

func (f FooerFunc) Foo(a A, b B) (C, D) {
    return f(a, b)
}

to use a closure to implement an interface. #21670 mentions http.HandlerFunc, but for me the cases that comes up most often are io.Reader and io.Writer, which I want to wrap with some trivial behavior. For example, if I want an io.Writer, which counts the numbers of bytes written, I can write

type countingWriter struct {
    w io.Writer
    n int64
}

func (w *countingWriter) Write(p []byte) (n int, err error) {
    n, err = w.w.Write(p)
    w.n += int64(n)
    return n, err
}

func main() {
    cw := &countingWriter{w: os.Stdout}
    // write things to cw
    fmt.Println(cw.n, "bytes written")
}

However, this makes the code look more heavyweight than it is, by adding a separate type, when really, the only really relevant line is w.n += int64(n). It also looks like it breaks encapsulation. (*countingWriter).n is used both by methods of the type and to communicate the end-result.

Compare this to

func main() {
    var N int64
    cw := io.Writer(func(p []byte) (n int, err error) {
        n, err = os.Stdout.Write(p)
        N += int64(n)
        return n, err
    })
    // write things to cw
    fmt.Println(N, "bytes written")
}

Here, the usage of a closure removes the whole consideration of encapsulation. It's not just less code - it's also code that is more localized and makes the interactions clearer.

We can get part of that currently, by implementing a type like http.HandlerFunc. But especially if it's only used once, that smells of over-abstraction and still has the same boiler-plate.

Proposal

I propose to add a bullet to the list of allowed conversions of a non-constant value to a type, saying (roughly):

T is an interface type containing exactly one method and x's type is identical to the type of the method value of Ts single method. The resulting T will call x to implement its method.

One way to implement this, would be for the compiler to automatically generate a (virtual) defined type, with an arbitrary unexported name in the runtime package.

Discussion

• The main difference to proposal: Go 2: Have functions auto-implement interfaces with only a single method of that same signature #21670 is that they propose assignability, while I propose convertibility. This is specifically to address a common concern about that proposal: io.Reader and io.Writer (for example) use the same signature for their method and it seems unsafe to have a function implement both, implicitly. This proposal addresses that concern, by requiring an explicit type-conversion - the programmer has to explicitly say if the function is intended to be an io.Reader or an io.Writer.
• The proposal suggests to generate a virtual defined type, so that no changes to tooling, reflect (except to allow the conversion itself) or the use of type-assertions is needed.
  • This handles the same as if, say, the io package defined an unexported type readerFunc func(p []byte) (int, error) implementing io.Reader and a func ReaderFunc(f func(p []byte) (int, error)) Reader { return readerFunc(f) }.
  • We don't have the mismatch of having methods on non-defined types: It's not func([]byte) (int, error) that carries the method, but runtime.io_reader.
  • type-assertion/switches to unpack the func are not possible, as the type is unexported. That is a plus, because again, the func type can't carry methods so it would be strange to use it in a type-assertion.
  • However, it is already possible to transform from var r io.Reader to a func([]byte) (int, error), by simply evaluating r.Read.
• The proposal suggest to put the generated type in the runtime package, because it seems like if two different packages do a func->interface conversion for the same interface, they should use the same type. I don't think the difference is observable though (the types are not comparable) so we could also put the type in the current package, similar to how closures turn up as pkgpath.func1 in stack-traces.
• In var f func(); g := Fooer(f).Foo, g should not be nil, even though f is. Method expressions are never nil, currently, and we should maintain this invariant. This requires that the compiler generates wrapper-methods.

---

Further, more general discussion can be found in #21670 - some arguments applying to that proposal, also apply here. As I said, I'm fine with this being closed as a duplicate, if it's deemed to similar - but I wanted to explicitly file this to increase the chance of moving forward.

[zephyrtronium]

The proposal suggests to generate a virtual defined type, so that no changes to tooling, reflect or the use of type-assertions is needed.

To be pedantic, this would require also changing package reflect, so that Value.Convert can convert function values to interface types when the language allows it. I don't remember whether this was mentioned in #21670.

[Merovius]

@zephyrtronium You are correct, forgot to mention that, fixed.

[D1CED]

If we already take two steps in this direction why not take the third one with #25860? Is there a good reason to artificially limit this feature to single method interfaces (for e.g. simpler implementation)?

[beoran]

When looking at the closure example above, I have to admit that in this case, this syntactic sugar would really help readability, because there is no need to go and read what the countingWriter is and does. All while the current level of type safety is maintained. So, I have to admit this makes me change my mind about this proposal.

However as you state :

This handles the same as if, say, the io package defined an unexported type readerFunc func(p []byte) (int, error) implementing io.Reader and a func ReaderFunc(f func(p []byte) (int, error)) Reader { return readerFunc(f) }

This points me to something I think is important. In stead of adding this conversion, There could be a convention in Go and iin the standard library where for every one-method reader, there should be an exported adapter function provided as you suggest. Of course, that would still have the performance cost of wrapping the function.

This brings me to why I still come to support this conversion proposal: it provides a possibility for improving performance by not having to wrap the closure in a struct and then unwrapping it again. In my experience closures are often used in this way, so it would help to avoid this wrapping.

About #25860: interface literals proposed there have a very serious problem with nil safety. One could easily fill in one of the multiple fields with a nil value, causing a panic when using the interface. AFAICS, this conversion here proposed, can be implemented in such a way that it does not have that problem. A conversion from a pointer-to-function to single method interface should be prohibited.

[Merovius]

@D1CED Personally, I don't find the idea of interface-literals very readable. YMMV, of course. If #25860 is adopted, that would likely make this not worth it. If it's rejected, this might still prove valuable.

[Merovius]

@beoran

interface literals proposed there have a very serious problem with nil safety. One could easily fill in one of the multiple fields with a nil value, causing a panic when using the interface. AFAICS, this conversion here proposed, can be implemented in such a way that it does not have that problem. A conversion from a pointer-to-function to single method interface should be prohibited.

I don't think the two proposals differ significantly in this regard. I would still allow converting a nil function to a single-value interface, which would then panic when called. We could make it so that using nil would result in a nil interface. IMO that's a bit of a strange special case though, for little benefit. Note that we are fine with code like this panicing, which is roughly equivalent:

type T struct{}

func (T) Foo() {}

type Fooer interface {
	Foo()
}

func main() {
	var f Fooer = (*T)(nil)
	f.Foo()
}

[beoran]

Well, for this proposal, at least we could limit the allowed conversion to the case of a closure as you use in your example, or perhaps anything the compiler can statically see that it cannot be nil.

As for the example you show, I consider that unfortunate. I am all in favor of making Go more nil safe whenever possible. So I'd like either proposal to do better than what we have now.

[Merovius]

Well, for this proposal, at least we could limit the allowed conversion to the case of a closure as you use in your example, or perhaps anything the compiler can statically see that it cannot be nil.

I am against doing that. We are doing a cost-benefit analysis and I'm only proposing this, because the cost is low. If we have to start distinguishing between different func values based on whether or not they are a function-literal and/or have to start describing heuristics based on which a compiler may prove that a value is nil, the complexity we add to the spec outweighs the benefit provided.

[beoran]

Well, yes, perhaps the cost does outweigh the benefit in this case. If I want a static nil checker for go, for this case, I should probably contribute to staticcheck (https://staticcheck.io/docs/checks).

[Merovius]

Based on the discussion in #25860 around nil methods, I've modified the proposal to answer the corresponding question here.

[carlmjohnson]

To be clear, I take it the behavior of g in var f func(); g := Fooer(f).Foo is the same as in this?:

type Fooer interface{ Foo() }
type FooFunc func()
func (f FooFunc) Foo() { f() }

func main() {
	var f func()
	g := FooFunc(f).Foo
	fmt.Println(f == nil, g == nil)
	g()
}

true false
panic: runtime error: invalid memory address or nil pointer dereference

[Merovius]

@carlmjohnson That would be my proposal, yes.

[rsc]

The spec says this about type assertions

In this case, T must implement the (interface) type of x

We'd have to change that to "must implement or be convertible to"

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[Merovius]

@rsc

The spec says this about type assertions

In this case, T must implement the (interface) type of x

We'd have to change that to "must implement or be convertible to"

The way I intended, this isn't needed. The dynamic type of the converted interface wouldn't be func(…) (…), but an unexported (autogenerated) defined type. So you can't type-assert, because you can't refer to that type. Instead, to go from a 1-interface method, to a func, you'd use a method value of that method.

[rsc]

The way I intended, this isn't needed. The dynamic type of the converted interface wouldn't be func(…) (…), but an unexported (autogenerated) defined type. So you can't type-assert, because you can't refer to that type. Instead, to go from a 1-interface method, to a func, you'd use a method value of that method.

If you know what you want to go to, that's fine. But if you are asking "what kind of implementation of io.Reader is this?" you want to be able to answer that question when the answer is "this was an ordinary function". At least I think you do.

[Merovius]

IMO there is a choice between a) don't make it possible to use type-assertions/-switches to "unpack" the interface, or b) violate the invariant that only defined types can have methods (possibly breaking code using reflect which assumes that this isn't possible) and add special cases in the spec for type-assertions/-switches on func, as you observe.

I agree that it's a downside not being able to find out what the dynamic type is. It just seems like the lesser evil. In particular as I don't think the usual use-cases for type-assertions/-switches really apply to this func case. ISTM that there are three general reasons you'd want to know the dynamic type:

1. You need the concrete value to pass to somewhere accepting the concrete type. In this case you can use a method value.
2. You want to specialize for performance or something. But calling the func shouldn't be any faster than calling the method, ISTM. So a default case calling the method (or using the method expression) should serve you just as well.
3. You want to use the interface as a sum-ish type. In that case, you can always define your own type and use that as the case.

But if you think supporting it is worth the cost, I won't argue against it. I suggested what seemed the more palatable, less intrusive choice, but I'd take the convenience no matter how it comes :)

[Merovius]

Oh and just to be safe: I assume it is a place-holder, but I don't think "must implement or be convertible to" is a good wording, either way. It would allow to type-assert something with underlying type float64 to int, which would be bad. If anything, we should specifically single out the func case.

[rsc]

Placed on hold.
— rsc for the proposal review group

[AndrewHarrisSPU]

I had an idea with syntax that is currently taboo:

Instead of:

reader := func( p []byte ) (n int, err error){
	return 0, io.EOF
}

io.ReadAll(io.Reader{reader})

s/func/interface.Ident/

interface.Ident is in most ways like a function literal.

r := interface.Read(p []byte) (int, error){
	return 0, io.EOF
}

// This works if, magically:
// - the `interface.Ident` expr results in a value is of some anonymous struct{} type
// - the anonymous struct{} has the method Read defined on it
// - (therefore, the anonymous struct{} satisfies io.Reader)
io.ReadAll( r )

I don't know that this is morally satisfying, but it feels scope-bound like a function literal or an anonymous struct. Really, we don't want an interface over state/symbols associated with an instance of a type, but an interface over the state/symbols associated with the current scope.

[Merovius]

@AndrewHarrisSPU That would probably work. Some observations:

• It would not allow using an already existing func (like a package-scoped function, or a variable) to be converted. That's not necessarily a bad thing (in fact, it would prevent some uncomfortable questions) but it should be mentioned.
• I think a nice aspect of only allowing 1-method interfaces is that it allows the dynamic type to be a function-type. The fact that we can't do that for proposal: Go 2: interface literals #25860 is one of the main reasons I'm not super convinced by that proposal. So, IMO, the dynamic type should not be a struct{}.
• Personally, I don't like the idea of requiring significant syntax-changes for this. What's more, the syntax you suggest is unlike anything we already have in Go. This is a minor language feature with few uses and having specialty syntax for it feels overkill to me. The main reason I like my proposal is that it fits pretty neatly into the existing language. We already have the conversion syntax, we already know how to talk about convertibility and what that means for the rest of the language.

The last point is really what leaves me unconvinced. Note that the primary issue with my proposal right now seems to be that we'd have to restrict the conversions to non-defined interface types and there not being a precedent in the language for conversions behaving differently between defined types and their underlying types. That feels to me like a relatively minor level of "not fitting well into the language", compared to making up an entirely new syntax and concept.

[AndrewHarrisSPU]

I don't at all want to suggest that function conversion isn't a promising idea here - personally I feel like, there are ways to do anonymous structs or functions, the ability to do something similarly in situ for interfaces IMHO is minor but also probably pretty commonly yearned for. Function conversion would handle a lot.

A very specific point/opinion I hold (not necessarily very strongly) is that it might be nice to have something where:

1. The syntax is a bit faster
2. A generated marker type - the resulting concrete/dynamic type, used for marking that the value is blessed as implementing the interface - has a zero value which behaves the same as the instance that generated it

The reasoning would be related to generics: if I'm not mistaken, it'd be possible to allocate new instances of the marker type even as its anonymity would make doing so implausible otherwise (excepting reflection). A struct{}'s zero value would be valid, while the func() sort of approach would nil out. As far as I've gotten in experiments with generics so far, I'm finding that some of the subtleties of granularly specifying what monomorphizes versus what uses interface dispatch, or how new instances of a type are created, can be subtle. I might just need more experience and perspective, but I do find myself reaching at structure where the idea is overriding a specific behavior, extending a generic type in a one-off kind of way - maybe more like detailing a class than a core type.

[Merovius]

The reasoning would be related to generics: if I'm not mistaken, it'd be possible to allocate new instances of the marker type even as its anonymity would make doing so implausible otherwise

I believe you are mistaken. You need to be able to either mention the type in the instantiation, which you can't because the generated type is transparent. Or it needs to get inferred by passing a variable of the specific type as an argument, which you can't have, because the resulting variable has type io.Reader (or whatever the interface is).

You can get the zero value of the dynamic type, using reflect. Personally, I think it's totally fair and expected for that to then panic. It would for http.HandlerFunc as well.

[AndrewHarrisSPU]

This is close to what I had scribbled out: https://go2goplay.golang.org/p/zoL14_rXGVT - I guess here I do need to know some undecorated T1 and the specific Fooer type T2 ... I think this smuggles a T2 in and out of Decorate(), with the help of yet another supporting type, such that I can dump out more of the decorated type.

I definitely respect the idea that maybe any in-situ interface conversion or generation really should just result in failure under elaboration like this.

[Merovius]

@AndrewHarrisSPU One significant difference is that in your code, fn has type fnfoo or structfoo, not Fooer, as it would be under the semantics of this proposal. fnfoo/structfoo would not be types which are syntactically accessible. So this is closer, if we use func and this is closer if we use struct{}. Both panic exactly the same, as you create a slice of nil-interfaces, which is expected.

So, as I said: Under the proposal, the actual dynamic type is not syntactically accessible and you can't declare a variable of its type, so it's not possible to instantiate a generic function/type using the dynamic type. But talking about generics is a distraction anyway. Generic code fundamentally can't give you access to more types than non-generic code - you can always inline the actual instantiations to end up with non-generic code.

And even if generics could somehow expand your capability to get a zero-value, it's not necessary - you can also use reflect to get one. So we don't need to dive into hypotheticals, this will always be a possibility.

[Merovius]

Another problem with making the underlying type struct{} is that those types are comparable and would always be equal. We probably don't want the result of the conversion to be comparable and we definitely don't want different function values to compare as equal after conversion. We might be able to hack around that, but it is icky and might be invasive for reflect and the like.

That is, while it might make sense to expose the type as if it's a struct{}, we should still consider that in actuality, the type does carry data - namely the relevant function pointer. Which seems a good indication that struct{} is not the right representation. We can use a single _-field, which would solve the expectations while still keeping the type opaque. But at that point, we lose the "the zero-value doesn't panic" property anyways and might as well just make it a func.

[Merovius]

@rsc Given that Go 1.18 is out, can we un-hold this?

[natefinch]

I feel like this could remove a lot of boilerplate and make code much easier to read.

from this:

mux.Handle("/ping", http.HandleFunc(someHandler))

to this:

mux.Handle("/ping", someHandler)

Also allows you to do stuff like this, without defining a whole type for it:

var Discard io.Writer = func(b []byte) (n, error) { return len(b), nil }

I really don't think that reflection should be a showstopper here. If you use reflection, you need to be prepared for your code to face challenges when the language changes, and this seems like a fairly unusual edge case for reflection to care about.

[Merovius]

@natefinch

from this: […] to this: […]

Note that the proposal is to still require a conversion. i.e. it's

mux.Handle("/ping", http.HandleFunc(someHandler))
// vs
mux.Handle("/ping", http.Handler(someHandler))

Which is less of a difference (which is why I don't think http.Handler is the best example - http.HandlerFunc already exists and does most of the heavy lifting).

[Merovius]

@rsc friendly ping? I don't know who else can/would make this decision?

[ianlancetaylor]

I'll take this off hold.

[rsc]

@robpike, @ianlancetaylor, @griesemer, and I all think this is a good change worth trying.
We should experiment with it before deciding whether to accept the proposal.

Does anyone want to prototype it?

[DmitriyMV]

type F func()
type (f F) M() { fmt.Println("F.M") }
type I interface { M() }
var FI F = func() { fmt.Println("FI") }
var V1 I = FI
var V2 = I(FI)
func main() {
    fmt.Printf("%T\n", V1) // prints I
    fmt.Printf("%T\n", V2) // prints I
    V1.M() // prints F.M
    V2.M() // prints F.M
}

After some additional tinkering I think current behavior is correct one, even if this proposal is accepted and implemented. Since F is named type with defined method M() there is no reason to upgrade function value to method M(). Mostly because it's already properly defined.

[rsc]

The ambiguity with existing valid programs definitely warrants more consideration.

Considering again #25860, it's true that there is not a lot of difference between:

io.Copy(w, io.Reader(myFunc))
io.Copy(w, io.Reader{Read: myFunc})

Maybe the more general form is better after all. We should clearly think more about this.

Either way, this is blocked on someone prototyping either or both of these.

[rsc]

Will put on hold for a prototype.

[rsc]

Placed on hold.
— rsc for the proposal review group

[meyermarcel]

I think the proposal is good if you take the writer's perspective, and I myself currently work in a language that allows the proposed construct. Would I like to write the longer 'heavyweight' example (second code excerpt from the description)? No, because a Closure would be significantly more practical to write.
Would I rather read the longer 'heavyweight' example than the shorter example with Closure? Yes, because everyone has written it that way so far and I can trust that it will always be written that way.
If this change were introduced, it would fragment the writing and change the readability because there is one more way to write the code.
I think the reason people come to Go is for readability. In almost all programming language designs, the emphasis is on the expressive power of the writer. Go is an exceptional exception (I must admit I don't know many programming languages).
Could it be better to accept less writing elegance to achieve better readability through consistency?
IMO readability of consistent code is one or maybe the biggest advantage of Go. Maybe we should look more at what disadvantage the writer's advantage brings to the reader. If you believe a well-known person from the IT scene, writing code and reading code are in a ratio of 1:10.