New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reflect: NamedOf #16522

Open
crawshaw opened this Issue Jul 28, 2016 · 15 comments

Comments

Projects
None yet
@crawshaw
Contributor

crawshaw commented Jul 28, 2016

Consider adding a function to the reflect package for creating a new named type with a method set:

func NamedOf(t Type, name string, methods []Method) Type

(Broken out from #4146.)

@crawshaw crawshaw added this to the Go1.8 milestone Jul 28, 2016

@crawshaw crawshaw self-assigned this Jul 28, 2016

@crawshaw

This comment has been minimized.

Contributor

crawshaw commented Aug 2, 2016

This turns out to require more invasive changes than I originally realized.

To be useful, it needs to be possible to add methods to types created at run time. That means taking some equivalent to func(receiver reflect.Value, args []reflect.Value) (results []reflect.Value) as the user's implementation of a method.

But methods are called directly from pointers to the text segment (stored in the itab). The unboxed receiver value is on the stack for the method body to use, but no type is available. So there's no way for the reflect package to take the receiver value and box it up into a reflect.Value when the method is called.

Assuming we want to avoid code generation, some change to dynamic method calling would have to be made. The smallest is probably dropping the underlying type and method number onto the stack. A more significant change would be making the function pointers in the itab indirect like func() pointers (https://golang.org/s/go11func). A third option would be a note on the itab saying these are func()-style pointers. Any of these actions incurs some cost on dynamic method calls.

Still pondering.

@glycerine

This comment has been minimized.

glycerine commented Dec 19, 2016

The unboxed receiver value is on the stack for the method body to use, but no type is available.

I thought the precise GC work back in Go1.4 meant all types on the stack are known to the runtime. Is that no longer true?

@randall77

This comment has been minimized.

Contributor

randall77 commented Dec 19, 2016

We only have pointer/nonpointer bits for each word on the stack. We do not have full type information.

@jimmyfrasche

This comment has been minimized.

Member

jimmyfrasche commented Aug 21, 2017

A discussion in the original thread starting at #4146 (comment) eventually honed in on really wanting something like described in this issue.

Referencing here because there is some discussion about how NamedOf/NewTypeOf would ideally behave to satisfy the goal of extending known interfaces without hiding unknown optional methods.

@nerdatmath

This comment has been minimized.

Contributor

nerdatmath commented Dec 27, 2017

Since NamedOf() knows the new type T that it is creating, couldn't it build a closure for each method, and store a pointer to the runtime._type for T in each closure? I'm assuming we can build a reflect.Value given an unboxed value and runtime._type.

Since there is no subtyping, we don't have to worry about the receiver's actual type being different from T (or *T), right?

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Dec 27, 2017

@nerdatmath The language doesn't permit methods to be closures. So, for simplicity, the same mechanism is used to pass the receiver value as is used to pass the pointer to the closure. For a method, we pass the receiver, and for a closure, we pass the closure pointer. We don't have a way to pass both.

@nerdatmath

This comment has been minimized.

Contributor

nerdatmath commented Dec 28, 2017

OK another shot. Suppose T is a concrete type, s is of type T, x is of type A, and m is a method of T accepting a single parameter of type A. From the programmer's perspective, calling s.m(x) is essentially the same as calling T.m(s, x). Does that correspondence extend down to the calling convention? When calling T.m the compiler knows the concrete type of s (and knows that T.m knows it too), so it doesn't need to pass the type info.

If so, we already have a way to create functions of type func(T, A): runtime.FuncOf. Could this or a similar function create method pointers that could be put in itabs? Please forgive me if I'm being hopelessly naive.

@edaniels

This comment has been minimized.

Contributor

edaniels commented Dec 28, 2017

@nerdatmath, I think the issue that you'll run into that @ianlancetaylor is referring to is that once you do create the struct/type of T, the code that compiler will generate for s.m(x) will behave such that if you try to interact with s as the expected receivever, you will encounter a nil pointer exception do to the nature of creating a closure and the calling conventions described in https://golang.org/s/go11func. The word passed along the stack in the call to the function will be the closure data pointer, not the receiver (s).

I believe the long standing calling conventions would need to modified or extended in order to support this. There's probably a safe way to extend the compiler to support a new calling convention. I went down a very deep rabbit hole trying to implement NamedOf using StructOf as a reference and hit this roadblock.

@glycerine

This comment has been minimized.

glycerine commented Dec 28, 2017

But methods are called directly from pointers to the text segment (stored in the itab). The unboxed receiver value is on the stack for the method body to use, but no type is available.

I thought the concrete type was pointed at by the other half of the interface value (https://research.swtch.com/interfaces). If one is calling through the interface value, would the type not be available? What am I missing?

@nerdatmath

This comment has been minimized.

Contributor

nerdatmath commented Dec 28, 2017

I dropped the idea of creating a closure. What does reflect.MakeFunc create? Not a closure but an actual function like TopLevel in https://golang.org/s/go11func, right? From that document:

Direct call of method. In order to use the same generated code for both an indirect call of a func value and for a direct call, the code generated for a method (both value and pointer receivers) is chosen to have the same calling convention as a top-level function with the receiver as a leading argument.

So essentially, creating a method should be the same as creating a top-level function, which is done by reflect.MakeFunc. My apologies for referring to it as runtime.FuncOf earlier.

@nerdatmath

This comment has been minimized.

Contributor

nerdatmath commented Dec 28, 2017

OK so I guess reflect.MakeFunc actually creates something like a func literal / closure. So I think I see the difficulty.

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Dec 28, 2017

@glycerine The interface value has the type, yes, but in the current calling convention we don't pass the entire interface value in a method call. We only pass the value pointer inside the interface value. We don't need to pass the type, since by definition a method knows which type it has been compiled for.

@glycerine

This comment has been minimized.

glycerine commented Dec 28, 2017

Thanks Ian.

Related discussion (for others catching up) from Carl Chatfield's proposal a couple years back, calling convention adaptation to provide the entire interface value.

https://groups.google.com/d/msg/golang-dev/coyYwxU3dfM/dZzOleaeFu4J

@cosmos72

This comment has been minimized.

cosmos72 commented May 13, 2018

In my opinion, reflect.NamedOf would be useful even without support for adding methods to the newly created type - just like reflect.StructOf is useful even without (full) support for wrapper methods of embedded fields.

In any case, I think that reflect.NamedOf() should have, as proposed, a function parameter to pass the list of methods and, until support for adding methods is implemented, it could simply panic if the methods list is non-empty.

I also have an idea on how to support adding methods to types created with reflect.NamedOf without changing the calling convention for methods, but it's quite tricky - better if I write a prototype first.

P.S. it is worth noting that the signature func NamedOf(t Type, name string, methods []Method) Type proposed above would not be able to create recursive types: they need a multi-step procedure, analogous to what go/types.Named has:

func NewNamed(obj *TypeName, underlying Type, methods []*Func) *Named
// then create an underlying type that contains the result of NewNamed
// and pass it to SetUnderlying:
func (t *Named) SetUnderlying(underlying Type)
@corebreaker

This comment has been minimized.

corebreaker commented Oct 30, 2018

In my opinion, reflect.NamedOf would be useful even without support for adding methods to the newly created type - just like reflect.StructOf is useful even without (full) support for wrapper methods of embedded fields.

In any case, I think that reflect.NamedOf() should have, as proposed, a function parameter to pass the list of methods and, until support for adding methods is implemented, it could simply panic if the methods list is non-empty.

I also have an idea on how to support adding methods to types created with reflect.NamedOf without changing the calling convention for methods, but it's quite tricky - better if I write a prototype first.

P.S. it is worth noting that the signature func NamedOf(t Type, name string, methods []Method) Type proposed above would not be able to create recursive types: they need a multi-step procedure, analogous to what go/types.Named has:

func NewNamed(obj *TypeName, underlying Type, methods []*Func) *Named
// then create an underlying type that contains the result of NewNamed
// and pass it to SetUnderlying:
func (t *Named) SetUnderlying(underlying Type)

Yes you are right, and it will introduce to the idea of Proxy Object.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment