reflect: NamedOf #16522

Open
crawshaw opened this Issue Jul 28, 2016 · 13 comments

Comments

Projects
None yet
10 participants
Member

crawshaw commented Jul 28, 2016

Consider adding a function to the reflect package for creating a new named type with a method set:

func NamedOf(t Type, name string, methods []Method) Type

(Broken out from #4146.)

@crawshaw crawshaw added this to the Go1.8 milestone Jul 28, 2016

@crawshaw crawshaw self-assigned this Jul 28, 2016

Member

crawshaw commented Aug 2, 2016

This turns out to require more invasive changes than I originally realized.

To be useful, it needs to be possible to add methods to types created at run time. That means taking some equivalent to func(receiver reflect.Value, args []reflect.Value) (results []reflect.Value) as the user's implementation of a method.

But methods are called directly from pointers to the text segment (stored in the itab). The unboxed receiver value is on the stack for the method body to use, but no type is available. So there's no way for the reflect package to take the receiver value and box it up into a reflect.Value when the method is called.

Assuming we want to avoid code generation, some change to dynamic method calling would have to be made. The smallest is probably dropping the underlying type and method number onto the stack. A more significant change would be making the function pointers in the itab indirect like func() pointers (https://golang.org/s/go11func). A third option would be a note on the itab saying these are func()-style pointers. Any of these actions incurs some cost on dynamic method calls.

Still pondering.

@quentinmit quentinmit added the NeedsFix label Oct 10, 2016

@rsc rsc modified the milestones: Go1.9, Go1.8 Oct 13, 2016

The unboxed receiver value is on the stack for the method body to use, but no type is available.

I thought the precise GC work back in Go1.4 meant all types on the stack are known to the runtime. Is that no longer true?

Contributor

randall77 commented Dec 19, 2016

We only have pointer/nonpointer bits for each word on the stack. We do not have full type information.

@crawshaw crawshaw modified the milestones: Go1.10, Go1.9 May 5, 2017

A discussion in the original thread starting at #4146 (comment) eventually honed in on really wanting something like described in this issue.

Referencing here because there is some discussion about how NamedOf/NewTypeOf would ideally behave to satisfy the goal of extending known interfaces without hiding unknown optional methods.

@bradfitz bradfitz modified the milestones: Go1.10, Unplanned Nov 29, 2017

Contributor

nerdatmath commented Dec 27, 2017

Since NamedOf() knows the new type T that it is creating, couldn't it build a closure for each method, and store a pointer to the runtime._type for T in each closure? I'm assuming we can build a reflect.Value given an unboxed value and runtime._type.

Since there is no subtyping, we don't have to worry about the receiver's actual type being different from T (or *T), right?

Contributor

ianlancetaylor commented Dec 27, 2017

@nerdatmath The language doesn't permit methods to be closures. So, for simplicity, the same mechanism is used to pass the receiver value as is used to pass the pointer to the closure. For a method, we pass the receiver, and for a closure, we pass the closure pointer. We don't have a way to pass both.

Contributor

nerdatmath commented Dec 28, 2017

OK another shot. Suppose T is a concrete type, s is of type T, x is of type A, and m is a method of T accepting a single parameter of type A. From the programmer's perspective, calling s.m(x) is essentially the same as calling T.m(s, x). Does that correspondence extend down to the calling convention? When calling T.m the compiler knows the concrete type of s (and knows that T.m knows it too), so it doesn't need to pass the type info.

If so, we already have a way to create functions of type func(T, A): runtime.FuncOf. Could this or a similar function create method pointers that could be put in itabs? Please forgive me if I'm being hopelessly naive.

edaniels commented Dec 28, 2017

@nerdatmath, I think the issue that you'll run into that @ianlancetaylor is referring to is that once you do create the struct/type of T, the code that compiler will generate for s.m(x) will behave such that if you try to interact with s as the expected receivever, you will encounter a nil pointer exception do to the nature of creating a closure and the calling conventions described in https://golang.org/s/go11func. The word passed along the stack in the call to the function will be the closure data pointer, not the receiver (s).

I believe the long standing calling conventions would need to modified or extended in order to support this. There's probably a safe way to extend the compiler to support a new calling convention. I went down a very deep rabbit hole trying to implement NamedOf using StructOf as a reference and hit this roadblock.

But methods are called directly from pointers to the text segment (stored in the itab). The unboxed receiver value is on the stack for the method body to use, but no type is available.

I thought the concrete type was pointed at by the other half of the interface value (https://research.swtch.com/interfaces). If one is calling through the interface value, would the type not be available? What am I missing?

Contributor

nerdatmath commented Dec 28, 2017

I dropped the idea of creating a closure. What does reflect.MakeFunc create? Not a closure but an actual function like TopLevel in https://golang.org/s/go11func, right? From that document:

Direct call of method. In order to use the same generated code for both an indirect call of a func value and for a direct call, the code generated for a method (both value and pointer receivers) is chosen to have the same calling convention as a top-level function with the receiver as a leading argument.

So essentially, creating a method should be the same as creating a top-level function, which is done by reflect.MakeFunc. My apologies for referring to it as runtime.FuncOf earlier.

Contributor

nerdatmath commented Dec 28, 2017

OK so I guess reflect.MakeFunc actually creates something like a func literal / closure. So I think I see the difficulty.

Contributor

ianlancetaylor commented Dec 28, 2017

@glycerine The interface value has the type, yes, but in the current calling convention we don't pass the entire interface value in a method call. We only pass the value pointer inside the interface value. We don't need to pass the type, since by definition a method knows which type it has been compiled for.

Thanks Ian.

Related discussion (for others catching up) from Carl Chatfield's proposal a couple years back, calling convention adaptation to provide the entire interface value.

https://groups.google.com/d/msg/golang-dev/coyYwxU3dfM/dZzOleaeFu4J

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment