cmd/gc: compiling a large []byte causes memory bloat in the compiler #6077

Closed
davecheney opened this Issue Aug 8, 2013 · 3 comments

Comments

Projects
None yet
4 participants
Contributor

davecheney commented Aug 8, 2013

What steps will reproduce the problem?

d91993212194 caused a regression in the compiler which causes it to use large amounts of
memory when compiling expressions like

        buf := [1<<30]byte{}


What is the expected output? What do you see instead?

Expected, before

lucky(~/go/test/fixedbugs) % /usr/bin/time -v ~/go/pkg/tool/linux_amd64/6g -o /tmp/foo
bug361.go
        Command being timed: "/home/dfc/go/pkg/tool/linux_amd64/6g -o /tmp/foo bug361.go"
        User time (seconds): 0.00
        System time (seconds): 0.00
        Percent of CPU this job got: 66%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.00
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 1776
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 501
        Voluntary context switches: 1
        Involuntary context switches: 2
        Swaps: 0
        File system inputs: 0
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

Got:

lucky(~/go/test/fixedbugs) % /usr/bin/time -v ~/go/pkg/tool/linux_amd64/6g -o /tmp/foo
bug361.go 
        Command being timed: "/home/dfc/go/pkg/tool/linux_amd64/6g -o /tmp/foo bug361.go"
        User time (seconds): 0.67
        System time (seconds): 0.19
        Percent of CPU this job got: 99%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.87
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 612352
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 153188
        Voluntary context switches: 1
        Involuntary context switches: 76
        Swaps: 0
        File system inputs: 0
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

Please use labels and text to provide additional information.

%  hg id
ecf3c8d0918c+ tip

This is related to issue #6023 and issue #6033. It was possibly exposed earlier by
https://code.google.com/p/go/source/detail?r=a5eb738ff6490af23ebf7906dd2af0f4bbc19a3e,
but was rolled back.

Comment 2:

The root of the problem seems to be in the Prog node allocations when generating the GC
information for function local variables.
==59552== 604,000,000 bytes in 1,208 blocks are possibly lost in loss record 50 of 50
==59552==    at 0x108F4EB: malloc (vg_replace_malloc.c:274)
==59552==    by 0x42F577: gethunk (subr.c:435)
==59552==    by 0x42F636: mal (subr.c:467)
==59552==    by 0x40B534: prog (gsubr.c:74)
==59552==    by 0x40B608: gins (gsubr.c:1038)
==59552==    by 0x408E3C: duintxx (gobj.c:464)
==59552==    by 0x403000: compile (pgen.c:351)
==59552==    by 0x41729C: funccompile (dcl.c:1453)
==59552==    by 0x4232A8: p9main (lex.c:425)
==59552==    by 0x450978: main (main.c:55)
Contributor

rsc commented Aug 8, 2013

Comment 3:

Owner changed to @rsc.

Status changed to Started.

Contributor

rsc commented Aug 8, 2013

Comment 4:

This issue was closed by revision 1f4d58a.

Status changed to Fixed.

@rsc rsc added this to the Go1.2 milestone Apr 14, 2015

@rsc rsc removed the go1.2 label Apr 14, 2015

@gopherbot gopherbot locked and limited conversation to collaborators Jun 24, 2016

This issue was closed.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.