cmd/gc: compiling a large []byte causes memory bloat in the compiler

[davecheney]

What steps will reproduce the problem?

d91993212194 caused a regression in the compiler which causes it to use large amounts of
memory when compiling expressions like

        buf := [1<<30]byte{}


What is the expected output? What do you see instead?

Expected, before

lucky(~/go/test/fixedbugs) % /usr/bin/time -v ~/go/pkg/tool/linux_amd64/6g -o /tmp/foo
bug361.go
        Command being timed: "/home/dfc/go/pkg/tool/linux_amd64/6g -o /tmp/foo bug361.go"
        User time (seconds): 0.00
        System time (seconds): 0.00
        Percent of CPU this job got: 66%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.00
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 1776
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 501
        Voluntary context switches: 1
        Involuntary context switches: 2
        Swaps: 0
        File system inputs: 0
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

Got:

lucky(~/go/test/fixedbugs) % /usr/bin/time -v ~/go/pkg/tool/linux_amd64/6g -o /tmp/foo
bug361.go 
        Command being timed: "/home/dfc/go/pkg/tool/linux_amd64/6g -o /tmp/foo bug361.go"
        User time (seconds): 0.67
        System time (seconds): 0.19
        Percent of CPU this job got: 99%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.87
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 612352
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 153188
        Voluntary context switches: 1
        Involuntary context switches: 76
        Swaps: 0
        File system inputs: 0
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

Please use labels and text to provide additional information.

%  hg id
ecf3c8d0918c+ tip

This is related to issue #6023 and issue #6033. It was possibly exposed earlier by
https://code.google.com/p/go/source/detail?r=a5eb738ff6490af23ebf7906dd2af0f4bbc19a3e,
but was rolled back.

[lexprfuncall]

Comment 2:

The root of the problem seems to be in the Prog node allocations when generating the GC
information for function local variables.
==59552== 604,000,000 bytes in 1,208 blocks are possibly lost in loss record 50 of 50
==59552==    at 0x108F4EB: malloc (vg_replace_malloc.c:274)
==59552==    by 0x42F577: gethunk (subr.c:435)
==59552==    by 0x42F636: mal (subr.c:467)
==59552==    by 0x40B534: prog (gsubr.c:74)
==59552==    by 0x40B608: gins (gsubr.c:1038)
==59552==    by 0x408E3C: duintxx (gobj.c:464)
==59552==    by 0x403000: compile (pgen.c:351)
==59552==    by 0x41729C: funccompile (dcl.c:1453)
==59552==    by 0x4232A8: p9main (lex.c:425)
==59552==    by 0x450978: main (main.c:55)

[rsc]

Comment 3:

Owner changed to @rsc.

Status changed to Started.

[rsc]

Comment 4:

This issue was closed by revision 1f4d58a.

Status changed to Fixed.