Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reflect: StructOf doesn't generate wrapper methods for embedded fields #15924

Open
adonovan opened this issue Jun 1, 2016 · 16 comments
Open

reflect: StructOf doesn't generate wrapper methods for embedded fields #15924

adonovan opened this issue Jun 1, 2016 · 16 comments
Labels
NeedsFix
Milestone

Comments

@adonovan
Copy link
Member

@adonovan adonovan commented Jun 1, 2016

% go version
go version devel +bbd1dcd Wed Jun 1 21:32:46 2016 +0000 linux/amd64
% cat ~/a.go
package main

import (
        "bytes"
        "io"
        "reflect"
        "sync"
)

func main() {
        t := reflect.StructOf([]reflect.StructField{
                {Type: reflect.TypeOf(sync.Mutex{}), Anonymous: true},
                {Type: reflect.TypeOf(bytes.Buffer{}), Anonymous: true},
        })
        x := reflect.New(t).Interface()
        _ = x.(io.Writer)
}

% go run ~/a.go
panic: interface conversion: *truct { sync.Mutex; bytes.Buffer } is not io.Writer: missing method Write

I expected that x would have generated wrapper methods for Mutex.Lock, etc.

Interestingly, adding this declaration inside func main (not at package level):

var _ interface{} = new(struct {
      sync.Mutex
      bytes.Buffer
})

causes the program to succeed. Presumably it causes the compiler to generate the correct type information, which reflect.StructOf then finds.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jun 2, 2016

This is a known deficiency, but it should be documented (or we should unexport StructOf until it is fixed).

CC @sbinet

@ianlancetaylor ianlancetaylor added this to the Go1.7 milestone Jun 2, 2016
@sbinet
Copy link
Member

@sbinet sbinet commented Jun 2, 2016

let me send a CL that documents this temporary (ie: go-1.7) limitation.

@gopherbot
Copy link

@gopherbot gopherbot commented Jun 2, 2016

CL https://golang.org/cl/23681 mentions this issue.

mk0x9 pushed a commit to mk0x9/go that referenced this issue Jun 2, 2016
This CL documents that StructOf currently does not generate wrapper
methods for embedded fields.

Updates golang#15924

Change-Id: I932011b1491d68767709559f515f699c04ce70d4
Reviewed-on: https://go-review.googlesource.com/23681
Reviewed-by: David Crawshaw <crawshaw@golang.org>
@ianlancetaylor ianlancetaylor added this to the Go1.8 milestone Jun 2, 2016
@ianlancetaylor ianlancetaylor removed this from the Go1.7 milestone Jun 2, 2016
@crawshaw
Copy link
Member

@crawshaw crawshaw commented Jul 27, 2016

cc me. I had a prototype of this that I didn't send out in the 1.7 cycle, hopefully I can get to it in 1.8.

@quentinmit quentinmit added the NeedsFix label Oct 10, 2016
@rsc rsc added this to the Go1.8Maybe milestone Oct 13, 2016
@rsc rsc removed this from the Go1.8 milestone Oct 13, 2016
@rsc
Copy link
Contributor

@rsc rsc commented Oct 20, 2016

Too late.

@rsc rsc added this to the Go1.9 milestone Oct 20, 2016
@rsc rsc removed this from the Go1.8Maybe milestone Oct 20, 2016
@crawshaw crawshaw added this to the Go1.10 milestone May 5, 2017
@crawshaw crawshaw removed this from the Go1.9 milestone May 5, 2017
@crawshaw
Copy link
Member

@crawshaw crawshaw commented May 5, 2017

Requires creating methods at run time, which is discussed in #16522.

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Jun 28, 2017

Dup with more repros in #20824

@gopherbot
Copy link

@gopherbot gopherbot commented Jun 28, 2017

CL https://golang.org/cl/47035 mentions this issue.

gopherbot pushed a commit that referenced this issue Jul 15, 2017
When StructOf is used with an anonymous field that has methods, and
that anonymous field is not the first field, the methods we generate
are incorrect because they do not offset to the field as required.
If we encounter that case, panic rather than doing the wrong thing.

Fixes #20824
Updates #15924

Change-Id: I3b0901ddbc6d58af5f7e84660b5e3085a431035d
Reviewed-on: https://go-review.googlesource.com/47035
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@rsc rsc removed this from the Go1.10 milestone Nov 29, 2017
@rsc rsc added this to the Go1.11 milestone Nov 29, 2017
@mdempsky
Copy link
Member

@mdempsky mdempsky commented Mar 15, 2018

FYI, as of CL 100846, methods are now sorted with exported names before non-exported names, and there's an extra field to track the number of exported methods.

The current code works because it only allows exported methods from one type, so they're already in the same sort order, and we don't need to worry about counting how many exported methods there are. If either of those constraints change, we'll need to change the code (e.g., to sort methods and/or count the exported methods). I've left TODOs to mark where I expect this needs to be done.

Also, one other comment: currently, StructOf only allows promoting methods from the first field, but it could be generalized slightly to allowing promoting methods of any field with offset==0. This would allow methods from multiple zero-width types to be promoted.

@gopherbot
Copy link

@gopherbot gopherbot commented Jun 27, 2018

Change https://golang.org/cl/121316 mentions this issue: reflect: prevent additional StructOf embedded method cases

gopherbot pushed a commit that referenced this issue Jun 27, 2018
The current implementation does not generate wrappers for methods of
embedded non-interface types. We can only skip the wrapper if
kindDirectIface of the generated struct type matches kindDirectIface
of the embedded type. Panic if that is not the case.

It would be better to actually generate wrappers, but that can be done
later.

Updates #15924
Fixes #24782

Change-Id: I01f5c76d9a07f44e1b04861bfe9f9916a04e65ca
Reviewed-on: https://go-review.googlesource.com/121316
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@cosmos72
Copy link

@cosmos72 cosmos72 commented Dec 26, 2018

while implementing reflect.NamedOf, I noticed that reflect.StructOf creates wrapper methods for embedded interfaces, and I was surprised that declaring methods at runtime would simply require calling reflect.MakeFunc (see https://github.com/golang/go/blob/master/src/reflect/type.go#L2386).

Alas, it seems not.

The following example crashes with SIGSEGV on both my local Go 1.11.4 installation and on playground https://play.golang.org/p/jny1YKZsHlp

package main

import (
	"fmt"
	"reflect"
)

type MyInt int

func (n MyInt) String() string {
	return fmt.Sprint("MyInt(value = ", int(n), ")")
}

func main() {
	embed()
	embed_reflect()
}

func embed() {
	type S = struct {
		fmt.Stringer
	}
	s := S{
		MyInt(3),
	}
	fmt.Println(s.Stringer.String()) // works, prints 3

	var i fmt.Stringer = s
	fmt.Println(i.String()) // works, prints 3
}

func embed_reflect() {
	tstringer := reflect.TypeOf((*fmt.Stringer)(nil)).Elem()

	t := reflect.StructOf(
		[]reflect.StructField{
			reflect.StructField{
				Name:      tstringer.Name(),
				Type:      tstringer,
				Anonymous: true,
			},
		},
	)
	v := reflect.New(t).Elem()
	v.Field(0).Set(reflect.ValueOf(MyInt(4)))

        i := v.Field(0).Interface().(fmt.Stringer)
	fmt.Println(i.String()) // works, prints 4

	i = v.Interface().(fmt.Stringer)
	fmt.Println(i.String()) // crashes
}

on my local Go 1.11.4, the output is:

MyInt(value = 3)
MyInt(value = 3)
MyInt(value = 4)
unexpected fault address 0xc00000a080
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x2 addr=0xc00000a080 pc=0xc00000a080]

goroutine 1 [running]:
runtime.throw(0x4eebf7, 0x5)
	/usr/local/go/src/runtime/panic.go:608 +0x72 fp=0xc000055e00 sp=0xc000055dd0 pc=0x4288a2
runtime.sigpanic()
	/usr/local/go/src/runtime/signal_unix.go:397 +0x275 fp=0xc000055e50 sp=0xc000055e00 pc=0x43b7f5
main.embed_reflect()
	/home/max/go/src/github.com/cosmos72/gomacro/_example/namedof.go:51 +0x386 fp=0xc000055f88 sp=0xc000055e50 pc=0x4b5486
main.main()
	/home/max/go/src/github.com/cosmos72/gomacro/_example/namedof.go:16 +0x25 fp=0xc000055f98 sp=0xc000055f88 pc=0x4b4f65
runtime.main()
	/usr/local/go/src/runtime/proc.go:201 +0x207 fp=0xc000055fe0 sp=0xc000055f98 pc=0x42a217
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1333 +0x1 fp=0xc000055fe8 sp=0xc000055fe0 pc=0x452a31

on playground https://play.golang.org/p/jny1YKZsHlp the output is instead:

MyInt(value = 3)
MyInt(value = 3)
MyInt(value = 4)
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0xffffffff addr=0x0 pc=0x40c0e0]

goroutine 1 [running]:
main.embed_reflect()
	/tmp/sandbox101416977/main.go:51 +0x400
main.main()
	/tmp/sandbox101416977/main.go:16 +0x40

I would expect that either reflect.StructOf fails (due to unsupported interface embedding) or that its returned type does not implement fmt.Stringer - I was surely not expecting that the method call crashes at runtime.

@mihaiav
Copy link

@mihaiav mihaiav commented Jan 18, 2019

83092a4#diff-cdbb65f700222dd514ff85e972c88be2R2470 broke my program after I've updated the Go version.

I didn't use any of the the struct methods. The struct was built only to create dynamic data structures matching a specific format when encoded with various encoders such encoding/json.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jan 18, 2019

@mihaiav I'm sorry that happened but given the choice between "incorrectly claim we did what the program requested" and "give an error" I think it's better to give an error.

You should be able to fix your program by making a regular field rather than an embedded field.

@lestrrat
Copy link

@lestrrat lestrrat commented Feb 12, 2020

I was reading https://groups.google.com/forum/#!topic/golang-nuts/e9ij7B_c5DE and thought this interesting, so I wrote some test cases: https://play.golang.org/p/Z93JIKEpkgv https://play.golang.org/p/C8nTXcQ7UUN

Seems like if
(1) The wrapper type is given as value
(2) The embedded type is a value
(3) But the method receiver (in the embedded type) is declared as a pointer

We cannot call the method from the embedded type.

mvertes added a commit to traefik/yaegi that referenced this issue Jun 28, 2020
When generating an interface wrapper, lookup existing wrappers by method
to get the one with the biggest set of methods implemented by interpreter.

A string method is also added to wrappers, in order to provide a string
representation of the interpreter value rather than the wrapper itself
(at least for %s and %v verbs).

This allows the runtime to pickup an interpreter method automatically
even if the conversion to the interface is not specified in the script. As
in Go spec, it is enough for the type to implement the required methods.

A current limitation is that only single wrappers can be instantiated,
not allowing to compose interfaces.

This limitation can be removed when the Go reflect issue
golang/go#15924 is fixed.

Fixes #435.
mvertes added a commit to traefik/yaegi that referenced this issue Jun 29, 2020
* fix: make interpreter methods discoverable by runtime

When generating an interface wrapper, lookup existing wrappers by method
to get the one with the biggest set of methods implemented by interpreter.

A string method is also added to wrappers, in order to provide a string
representation of the interpreter value rather than the wrapper itself
(at least for %s and %v verbs).

This allows the runtime to pickup an interpreter method automatically
even if the conversion to the interface is not specified in the script. As
in Go spec, it is enough for the type to implement the required methods.

A current limitation is that only single wrappers can be instantiated,
not allowing to compose interfaces.

This limitation can be removed when the Go reflect issue
golang/go#15924 is fixed.

Fixes #435.

* test: add a simpler test
@dxq174510447
Copy link

@dxq174510447 dxq174510447 commented Apr 20, 2021

while implementing reflect.NamedOf, I noticed that reflect.StructOf creates wrapper methods for embedded interfaces, and I was surprised that declaring methods at runtime would simply require calling reflect.MakeFunc (see https://github.com/golang/go/blob/master/src/reflect/type.go#L2386).

Alas, it seems not.

The following example crashes with SIGSEGV on both my local Go 1.11.4 installation and on playground https://play.golang.org/p/jny1YKZsHlp

package main

import (
	"fmt"
	"reflect"
)

type MyInt int

func (n MyInt) String() string {
	return fmt.Sprint("MyInt(value = ", int(n), ")")
}

func main() {
	embed()
	embed_reflect()
}

func embed() {
	type S = struct {
		fmt.Stringer
	}
	s := S{
		MyInt(3),
	}
	fmt.Println(s.Stringer.String()) // works, prints 3

	var i fmt.Stringer = s
	fmt.Println(i.String()) // works, prints 3
}

func embed_reflect() {
	tstringer := reflect.TypeOf((*fmt.Stringer)(nil)).Elem()

	t := reflect.StructOf(
		[]reflect.StructField{
			reflect.StructField{
				Name:      tstringer.Name(),
				Type:      tstringer,
				Anonymous: true,
			},
		},
	)
	v := reflect.New(t).Elem()
	v.Field(0).Set(reflect.ValueOf(MyInt(4)))

        i := v.Field(0).Interface().(fmt.Stringer)
	fmt.Println(i.String()) // works, prints 4

	i = v.Interface().(fmt.Stringer)
	fmt.Println(i.String()) // crashes
}

on my local Go 1.11.4, the output is:

MyInt(value = 3)
MyInt(value = 3)
MyInt(value = 4)
unexpected fault address 0xc00000a080
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x2 addr=0xc00000a080 pc=0xc00000a080]

goroutine 1 [running]:
runtime.throw(0x4eebf7, 0x5)
	/usr/local/go/src/runtime/panic.go:608 +0x72 fp=0xc000055e00 sp=0xc000055dd0 pc=0x4288a2
runtime.sigpanic()
	/usr/local/go/src/runtime/signal_unix.go:397 +0x275 fp=0xc000055e50 sp=0xc000055e00 pc=0x43b7f5
main.embed_reflect()
	/home/max/go/src/github.com/cosmos72/gomacro/_example/namedof.go:51 +0x386 fp=0xc000055f88 sp=0xc000055e50 pc=0x4b5486
main.main()
	/home/max/go/src/github.com/cosmos72/gomacro/_example/namedof.go:16 +0x25 fp=0xc000055f98 sp=0xc000055f88 pc=0x4b4f65
runtime.main()
	/usr/local/go/src/runtime/proc.go:201 +0x207 fp=0xc000055fe0 sp=0xc000055f98 pc=0x42a217
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1333 +0x1 fp=0xc000055fe8 sp=0xc000055fe0 pc=0x452a31

on playground https://play.golang.org/p/jny1YKZsHlp the output is instead:

MyInt(value = 3)
MyInt(value = 3)
MyInt(value = 4)
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0xffffffff addr=0x0 pc=0x40c0e0]

goroutine 1 [running]:
main.embed_reflect()
	/tmp/sandbox101416977/main.go:51 +0x400
main.main()
	/tmp/sandbox101416977/main.go:16 +0x40

I would expect that either reflect.StructOf fails (due to unsupported interface embedding) or that its returned type does not implement fmt.Stringer - I was surely not expecting that the method call crashes at runtime.

i meet the same question, how to solve it at last ?

@cosmos72
Copy link

@cosmos72 cosmos72 commented Apr 20, 2021

I am not aware of any general mechanism to create methods at runtime - if I remember correctly, it would require code generation.

The function reflect.StructOf can "create" - actually "pretend to create" - only wrapper methods for some special cases that allow reusing the existing methods of embedded fields.

And that's not the only difficulty that reflect.NamedOf is expected to solve: there's also the issue of recursive types - see issue #39717

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsFix
Projects
None yet
Development

No branches or pull requests