Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/gc: compiling a large []byte causes memory bloat in the compiler #6077

Closed
davecheney opened this issue Aug 8, 2013 · 3 comments
Closed

cmd/gc: compiling a large []byte causes memory bloat in the compiler #6077

davecheney opened this issue Aug 8, 2013 · 3 comments
Assignees
Milestone

Comments

@davecheney
Copy link
Contributor

@davecheney davecheney commented Aug 8, 2013

What steps will reproduce the problem?

d91993212194 caused a regression in the compiler which causes it to use large amounts of
memory when compiling expressions like

        buf := [1<<30]byte{}


What is the expected output? What do you see instead?

Expected, before

lucky(~/go/test/fixedbugs) % /usr/bin/time -v ~/go/pkg/tool/linux_amd64/6g -o /tmp/foo
bug361.go
        Command being timed: "/home/dfc/go/pkg/tool/linux_amd64/6g -o /tmp/foo bug361.go"
        User time (seconds): 0.00
        System time (seconds): 0.00
        Percent of CPU this job got: 66%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.00
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 1776
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 501
        Voluntary context switches: 1
        Involuntary context switches: 2
        Swaps: 0
        File system inputs: 0
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

Got:

lucky(~/go/test/fixedbugs) % /usr/bin/time -v ~/go/pkg/tool/linux_amd64/6g -o /tmp/foo
bug361.go 
        Command being timed: "/home/dfc/go/pkg/tool/linux_amd64/6g -o /tmp/foo bug361.go"
        User time (seconds): 0.67
        System time (seconds): 0.19
        Percent of CPU this job got: 99%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.87
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 612352
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 153188
        Voluntary context switches: 1
        Involuntary context switches: 76
        Swaps: 0
        File system inputs: 0
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

Please use labels and text to provide additional information.

%  hg id
ecf3c8d0918c+ tip

This is related to issue #6023 and issue #6033. It was possibly exposed earlier by
https://code.google.com/p/go/source/detail?r=a5eb738ff6490af23ebf7906dd2af0f4bbc19a3e,
but was rolled back.
@lexprfuncall

This comment has been minimized.

Copy link

@lexprfuncall lexprfuncall commented Aug 8, 2013

Comment 2:

The root of the problem seems to be in the Prog node allocations when generating the GC
information for function local variables.
==59552== 604,000,000 bytes in 1,208 blocks are possibly lost in loss record 50 of 50
==59552==    at 0x108F4EB: malloc (vg_replace_malloc.c:274)
==59552==    by 0x42F577: gethunk (subr.c:435)
==59552==    by 0x42F636: mal (subr.c:467)
==59552==    by 0x40B534: prog (gsubr.c:74)
==59552==    by 0x40B608: gins (gsubr.c:1038)
==59552==    by 0x408E3C: duintxx (gobj.c:464)
==59552==    by 0x403000: compile (pgen.c:351)
==59552==    by 0x41729C: funccompile (dcl.c:1453)
==59552==    by 0x4232A8: p9main (lex.c:425)
==59552==    by 0x450978: main (main.c:55)
@rsc

This comment has been minimized.

Copy link
Contributor

@rsc rsc commented Aug 8, 2013

Comment 3:

Owner changed to @rsc.

Status changed to Started.

@rsc

This comment has been minimized.

Copy link
Contributor

@rsc rsc commented Aug 8, 2013

Comment 4:

This issue was closed by revision 1f4d58a.

Status changed to Fixed.

@rsc rsc removed the go1.2 label Apr 14, 2015
@golang golang locked and limited conversation to collaborators Jun 24, 2016
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
4 participants
You can’t perform that action at this time.