cmd/cgo: godefs intermittently panics in go/printer

[wathiede]

Present on:

go version devel +52e26bb34741 Sun Aug 04 23:32:40 2013 +0400 freebsd/amd64

and

go version go1.1.1 freebsd/amd64

What steps will reproduce the problem?
This doesn't run on the playground as it is a go tool issue; the input source is:
http://play.golang.org/p/eZMYWmns3n
1. Run go tool cgo -godefs types_freebsd.go


What is the expected output?
Sometimes this output (which I think is correct) is produced:

// Created by cgo -godefs - DO NOT EDIT
// cgo -godefs types_freebsd.go

package main

type Xtcpcb struct {
        Len     uint64
        Inp     Inpcb
        Tp      _Ctype_struct_tcpcb
        Socket  _Ctype_struct_xsocket
        Alignment_hack  uint64
}
type Inpcb struct {
        Hash    _Ctype_struct___4
        List    _Ctype_struct___5
        Ppcb    *byte
        Pcbinfo *Inpcbinfo
        Socket  *Socket
        Cred    *Ucred
        Flow    uint32
        Flags   int32
        Flags2  int32
        Vflag   uint8
        Ip_ttl  uint8
        Ip_p    uint8
        Ip_minttl       uint8
        Flowid  uint32
        Refcount        uint32
        Pspare  [4]unsafe.Pointer
        Ispare  [4]uint32
        Inc     InConninfo
        Label   *[0]byte
        Sp      *[0]byte
        Depend4 _Ctype_struct___9
        Depend6 _Ctype_struct___10
        Portlist        _Ctype_struct___11
        Phd     *_Ctype_struct_inpcbport
        Gencnt  uint64
        Lle     *[0]byte
        Rt      *[0]byte
        Lock    Rwlock
}
type Inpcbinfo struct {
        Listhead        *_Ctype_struct_inpcbhead
        Count   uint32
        Pad_cgo_0       [4]byte
        Hashbase        *_Ctype_struct_inpcbhead
        Hashmask        uint64
        Porthashbase    *_Ctype_struct_inpcbporthead
        Porthashmask    uint64
        Lastport        uint16
        Lastlow uint16
        Lasthi  uint16
        Pad_cgo_1       [2]byte
        Zone    *[0]byte
        Gencnt  uint64
        Lock    Rwlock
        Vnet    *[0]byte
        Pspare  [2]unsafe.Pointer
}
 
type InConninfo struct {
        Flags   uint8
        Len     uint8
        Fibnum  uint16
        Ie      InEndpoints
}
type InEndpoints struct {
        Fport   uint16
        Lport   uint16
        Dependfaddr     [16]byte
        Dependladdr     [16]byte
}
type Socket struct {
        Count   int32
        Type    int16
        Options int16
        Linger  int16
        State   int16
        Qstate  int32
        Pcb     *byte
        Vnet    *[0]byte
        Proto   *[0]byte
        Head    *Socket
        Incomp  _Ctype_struct___2
        Comp    _Ctype_struct___3
        List    _Ctype_struct___4
        Qlen    uint16
        Incqlen uint16
        Qlimit  uint16
        Timeo   int16
        Error   uint16
        Pad_cgo_0       [6]byte
        Sigio   *[0]byte
        Oobmark uint64
        Aiojobq _Ctype_struct___5
        Rcv     _Ctype_struct_sockbuf
        Snd     _Ctype_struct_sockbuf
        Cred    *Ucred
        Label   *[0]byte
        Peerlabel       *[0]byte
        Gencnt  uint64
        Emuldata        *byte
        Accf    *_Ctype_struct_so_accf
        Fibnum  int32
        Pad_cgo_1       [4]byte
}
type Ucred struct {
        Ref     uint32
        Uid     uint32
        Ruid    uint32
        Svuid   uint32
        Ngroups int32
        Rgid    uint32
        Svgid   uint32
        Pad_cgo_0       [4]byte
        Uidinfo *[0]byte
        Ruidinfo        *[0]byte
        Prison  *[0]byte
        Pspare  *byte
        Flags   uint32
        Pad_cgo_1       [4]byte
        Pspare2 [2]unsafe.Pointer
        Label   *[0]byte
        Audit   AuditinfoAddr
        Groups  *uint32
        Agroups int32
        Pad_cgo_2       [4]byte
}
type Rwlock struct {
        Lock_object     LockObject
        Rw_lock uint64
}
type LockObject struct {
        Name    *int8
        Flags   uint32
        Data    uint32
        Witness *[0]byte
}
type AuditinfoAddr struct {
        Auid    uint32
        Mask    AuMask
        Termid  AuTidAddr
        Asid    int32
        Flags   uint64
}
type AuMask struct {
        Success uint32
        Failure uint32
}
type AuTidAddr struct {
        Port    uint32
        Type    uint32
        Addr    [4]uint32
}

const SizeofXtcpcb = 0x448
const SizeofInpcb = 0x150
const SizeofInpcbinfo = 0x80
const SizeofUcred = 0xa0




What do you see instead?
Sometimes I get:
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x28 pc=0x4c73f3]

goroutine 1 [running]:
go/printer.(*printer).expr1(0xc200197780, 0x0, 0x0, 0x0, 0x1, ...)
        /home/wathiede/Downloads/go/src/pkg/go/printer/nodes.go:676 +0x33
go/printer.(*printer).expr(0xc200197780, 0x0, 0x0)
        /home/wathiede/Downloads/go/src/pkg/go/printer/nodes.go:883 +0x4b
go/printer.(*printer).expr1(0xc200197780, 0xc200178240, 0xc200182e80, 0x0, 0x1, ...)
        /home/wathiede/Downloads/go/src/pkg/go/printer/nodes.go:708 +0x2771
go/printer.(*printer).expr(0xc200197780, 0xc200178240, 0xc200182e80)
        /home/wathiede/Downloads/go/src/pkg/go/printer/nodes.go:883 +0x4b
go/printer.(*printer).fieldList(0xc200197780, 0xc2001839c0, 0x1)
        /home/wathiede/Downloads/go/src/pkg/go/printer/nodes.go:442 +0xb11
go/printer.(*printer).expr1(0xc200197780, 0xc200178600, 0xc200182f60, 0x0, 0x1, ...)
        /home/wathiede/Downloads/go/src/pkg/go/printer/nodes.go:842 +0x2bf3
go/printer.(*printer).expr(0xc200197780, 0xc200178600, 0xc200182f60)
        /home/wathiede/Downloads/go/src/pkg/go/printer/nodes.go:883 +0x4b
go/printer.(*printer).spec(0xc200197780, 0xc200077f00, 0xc200061bd0, 0x1, 0x810719101,
...)
        /home/wathiede/Downloads/go/src/pkg/go/printer/nodes.go:1349 +0x862
go/printer.(*printer).genDecl(0xc200197780, 0xc2000a26c0)
        /home/wathiede/Downloads/go/src/pkg/go/printer/nodes.go:1394 +0x820
go/printer.(*printer).decl(0xc200197780, 0xc200077e40, 0xc2000a26c0)
        /home/wathiede/Downloads/go/src/pkg/go/printer/nodes.go:1522 +0xf8
go/printer.(*printer).declList(0xc200197780, 0xc2000a3600, 0x10, 0x20)
        /home/wathiede/Downloads/go/src/pkg/go/printer/nodes.go:1565 +0x172
go/printer.(*printer).file(0xc200197780, 0xc20009c500)
        /home/wathiede/Downloads/go/src/pkg/go/printer/nodes.go:1573 +0x16d
go/printer.(*printer).printNode(0xc200197780, 0x582340, 0xc20009c500, 0xc2001a9800,
0x43ac9f, ...)
        /home/wathiede/Downloads/go/src/pkg/go/printer/printer.go:1026 +0x7b2
go/printer.(*Config).fprint(0x69e5b0, 0xc2000ab2a0, 0xc20018ec40, 0xc200077480,
0x582340, ...)
        /home/wathiede/Downloads/go/src/pkg/go/printer/printer.go:1165 +0xa6
go/printer.(*Config).Fprint(0x69e5b0, 0xc2000ab2a0, 0xc20018ec40, 0xc200077480,
0x582340, ...)
        /home/wathiede/Downloads/go/src/pkg/go/printer/printer.go:1223 +0x7c
main.(*Package).godefs(0xc20007e750, 0xc20009c000, 0x7fffffffea24, 0x10, 0x0, ...)
        /home/wathiede/Downloads/go/src/cmd/cgo/godefs.go:112 +0xd81
main.main()
        /home/wathiede/Downloads/go/src/cmd/cgo/main.go:274 +0x10dc



Which compiler are you using (5g, 6g, 8g, gccgo)?
cgo built for FreeBSD/amd64

Which operating system are you using?
$  uname -a 
FreeBSD sagan.sf.xinu.tv 8.3-RELEASE-p9 FreeBSD 8.3-RELEASE-p9 #0: Fri Jul 26 23:07:20
UTC 2013     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64



Which version are you using?  (run 'go version')
Present in:

 go version devel +52e26bb34741 Sun Aug 04 23:32:40 2013 +0400 freebsd/amd64

and

  go version go1.1.1 freebsd/amd64

Please provide any additional information below.

[rsc]

Comment 1:

Labels changed: added priority-later, removed priority-triage.

Status changed to Accepted.

[rsc]

Comment 2:

The crash trace shows the printer printing a StarExpr with no operand (x.X == nil).
That cannot happen in the ASTs that cgo -godefs generates.
It suggests a garbage collection problem.
If you run it 10 or 100 times in a row, how often does it crash?
If you export GOGC=off does the crash probability change?
If you export GOGC=1 does the crash probability change?
for i in $(seq 100); do go tool cgo -godefs x.go >/dev/null 2>/dev/null || echo BAD $i;
done
I cannot reproduce the crash with a similar input file on OS X. I don't have FreeBSD at
hand.
I'd like to track this down but I am not 100% sure it will happen for Go 1.2.
Leaving as Go1.2Maybe for now.

[wathiede]

Comment 3:

For the record I'm using go version go1.1.2 freebsd/amd64.
It happens ~85% of the time invariant of the GOGC setting:
$ export GOGC=1; for ((i=0; i<100; i++)); do  go tool cgo -godefs
types_freebsd.go>/dev/null 2>/dev/null || echo BAD $i; done|wc -l
      85
$ export GOGC=off; for ((i=0; i<100; i++)); do  go tool cgo -godefs
types_freebsd.go>/dev/null 2>/dev/null || echo BAD $i; done|wc -l
      87
$ export GOGC=100; for ((i=0; i<100; i++)); do  go tool cgo -godefs
types_freebsd.go>/dev/null 2>/dev/null || echo BAD $i; done|wc -l
      84

[rsc]

Comment 4:

This is going to have to wait until Go 1.3.

Labels changed: added go1.3, removed go1.2maybe.

[rsc]

Comment 5:

Labels changed: added release-go1.3.

[rsc]

Comment 6:

Labels changed: removed go1.3.

[rsc]

Comment 7:

Labels changed: added repo-main.

[rsc]

Comment 8:

I still don't have a way to reproduce this. Until we do it is not scheduled for fixing
in any release.

Labels changed: added suggested, removed release-go1.3.

[rsc]

Comment 9:

Labels changed: added release-none.

[mdempsky]

Comment 10:

Hi couchmo....  I think this might be a duplicate of issue #8368, which was fixed by
revision 0015a25415455c390ae135bef7de2d5e2b10ee88.  I'd appreciate if you could repeat
your testing against Go tip and let me know if you're still able to reproduce the
intermittent panics.
If so, it would be helpful if you could attach the output of running "gcc -E" against
your Cgo preamble so we can try to reproduce the issue on non-FreeBSD.  Also if you
could try to minimize the test case.  I'd suspect the root cause was the
LIST_HEAD/LIST_ENTRY macros.

[mdempsky]

Comment 11:

Starting from revision 52e26bb34741, I was able to reproduce a very similar stack trace
by running "go tool cgo -godefs issue6040.go" on linux/amd64 using the below Go snippet.
 I can also reproduce the issue immediately before revision 0015a254, but not at or
after.
So I believe this issue is now fixed.
package issue6040
/*
typedef struct one one;
typedef struct two two;
typedef struct three three;
struct one {
        two *x;
        three *y;
};
struct two {
        one *x;
        three *y;
};
struct three {
        one *x;
        two *y;
};
*/
import "C"
type One C.one
type Two C.two
type Three C.three

[ianlancetaylor]

Comment 12:

Thanks for checking this.

Status changed to Fixed.