New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: 1.7 compiler does not compile program with large complex struct #18920

Open
rdvorak opened this Issue Feb 3, 2017 · 12 comments

Comments

Projects
None yet
10 participants
@rdvorak

rdvorak commented Feb 3, 2017

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.7.5 windows/amd64

What operating system and processor architecture are you using (go env)?

Windows Server, Intel

What did you do?

I want to compile a program with large complex struct.
Go 1.7.1, 1.7.5 does not compile the program but the version 1.6.4 does and
it works (Un/Marshalling) perfectly with 1.6.
I have created repo with single main.go file just for this bug report.
https://github.com/rdvorak/largeGoStruct/blob/master/main.go
I had to hash identifiers in the struct because the real structure should not be published.

What did you expect to see?

I want the main.go get compiled.

What did you see instead?

.\main.go:1: internal compiler error: name too long: *struct
.. follows the large struct ...
goroutine 1 [running]:
runtime/debug.Stack(0x0, 0x0, 0x0)
c:/go/src/runtime/debug/stack.go:24 +0x80
cmd/compile/internal/gc.Fatalf(0x911ba1, 0x11, 0xc044fecd20, 0x1, 0x1)
c:/go/src/cmd/compile/internal/gc/subr.go:165 +0x24f
cmd/compile/internal/gc.dnameData(0xc043f2c120, 0x0, 0xc044860000, 0x29331, 0x0, 0x0, 0x0, 0x0, 0x0)
c:/go/src/cmd/compile/internal/gc/reflect.go:515 +0x444
cmd/compile/internal/gc.dname(0xc044860000, 0x29331, 0x0, 0x0, 0x0, 0xc044860000, 0x29331)
c:/go/src/cmd/compile/internal/gc/reflect.go:583 +0x1c8
cmd/compile/internal/gc.dcommontype(0xc042b54000, 0x28, 0xc0426e6a50, 0x9043ff)
c:/go/src/cmd/compile/internal/gc/reflect.go:911 +0x4cb
cmd/compile/internal/gc.dtypesym(0xc0426e6a50, 0xc0426e6a50)
c:/go/src/cmd/compile/internal/gc/reflect.go:1300 +0xef1
cmd/compile/internal/gc.dtypesym(0xc0427306e0, 0xc0426c10a0)
c:/go/src/cmd/compile/internal/gc/reflect.go:1284 +0xd3e
cmd/compile/internal/gc.dtypesym(0xc0426e69b0, 0xc042732380)
c:/go/src/cmd/compile/internal/gc/reflect.go:1296 +0xe98
cmd/compile/internal/gc.dumptypestructs()
c:/go/src/cmd/compile/internal/gc/reflect.go:1369 +0xd3
cmd/compile/internal/gc.dumpobj1(0xc0420440c0, 0x3a, 0x3)
c:/go/src/cmd/compile/internal/gc/obj.go:133 +0x566
cmd/compile/internal/gc.dumpobj()
c:/go/src/cmd/compile/internal/gc/obj.go:45 +0x59
cmd/compile/internal/gc.Main()
c:/go/src/cmd/compile/internal/gc/main.go:490 +0x1aa6
cmd/compile/internal/amd64.Main()
c:/go/src/cmd/compile/internal/amd64/galign.go:93 +0x301
main.main()
c:/go/src/cmd/compile/main.go:33 +0x2aa

@ianlancetaylor ianlancetaylor changed the title from 1.7 compiler does not compile program with large complex struct to cmd/compile: 1.7 compiler does not compile program with large complex struct Feb 3, 2017

@ianlancetaylor ianlancetaylor added this to the Go1.9 milestone Feb 3, 2017

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Feb 3, 2017

@crawshaw

This comment has been minimized.

Contributor

crawshaw commented Feb 3, 2017

if len(name) > 1<<16-1 {
        Fatalf("name too long: %s", name)
}

That's a big name!

@LionNatsu

This comment has been minimized.

Contributor

LionNatsu commented Feb 6, 2017

I think we should call it 'type name' in error message rather than call it 'name' which makes me feel that was a variable name…?

@odeke-em

This comment has been minimized.

Member

odeke-em commented Feb 11, 2017

I don't think this bug only applies to Go1.7, I can reproduce the message on >Go1.8 ie +3a6842a

$ go version
go version devel +3a6842a Thu Feb 9 04:13:35 2017 +0000 darwin/amd64

so I'll just do a minor retitling of the bug.

@odeke-em odeke-em changed the title from cmd/compile: 1.7 compiler does not compile program with large complex struct to cmd/compile: improve type name error message on failing to compile very large complex struct Feb 11, 2017

@pborges

This comment has been minimized.

pborges commented Feb 27, 2017

Ouch... this bug just bit me bad, I am writing a package to generate some structs from a very complicated XSD, is there a work round other then attempting to simplify my structs so they are not so large?

And just to clarify, the problem is not the actual Name of my struct right? My name is only 25 characters

@rdvorak rdvorak changed the title from cmd/compile: improve type name error message on failing to compile very large complex struct to cmd/compile: 1.7 compiler does not compile program with large complex struct Feb 28, 2017

@rdvorak

This comment has been minimized.

rdvorak commented Feb 28, 2017

the workaround is to use 1.6 compiler or if you have large nested structs you can define them separately

@crawshaw

This comment has been minimized.

Contributor

crawshaw commented Feb 28, 2017

Yes this is a real bug that should be fixed at some point. But I would also ask you to consider the readability of your generated code. Go code should be designed for reading, even when made by a machine. Are enormous nested types readable? Would intermediate naming help? (I suspect that almost always, it would.)

@rdvorak

This comment has been minimized.

rdvorak commented Feb 28, 2017

In my case the structure is a container for one record of related data and the schema, complexity is not my decision, as it is defined by external data provider- it is real world example.

@pborges

This comment has been minimized.

pborges commented Feb 28, 2017

@crawshaw You are probably right however the code I've generated looks pretty good, it has a bunch of nested structs however they are all only a few fields each, I can probably break them up into structs, I just need to come up with some naming scheme to avoid duplicate struct names :( Is there any information as to why this became a problem in 1.7+ (just curious)

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Feb 28, 2017

The problem appeared in 1.7 because the type descriptors generated by the compiler were changed to make them smaller (for issue #6853). One of the changes was to store the length of the result of reflect.Type.String in two bytes rather than eight, leading to this error. To fix this we'll need some sort of escape mechanism for longer strings.

@pborges

This comment has been minimized.

pborges commented Feb 28, 2017

@ianlancetaylor Thank you for the information

@josharian

This comment has been minimized.

Contributor

josharian commented May 11, 2017

@crawshaw any plans to work on this for 1.9?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment