New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
all: reduce init-time CPU & memory usage #26775
Comments
Change https://golang.org/cl/127655 mentions this issue: |
Defer call to registerBasics until the first invocation of NewDecoder() or NewEncoder() This saves ~13.5kB of allocation in init() Updates golang#26775
Change https://golang.org/cl/127660 mentions this issue: |
Change https://golang.org/cl/127661 mentions this issue: |
Change https://golang.org/cl/127664 mentions this issue: |
Change https://golang.org/cl/127735 mentions this issue: |
Change https://golang.org/cl/127736 mentions this issue: |
Wow, that was picked up real fast by the others! |
Change https://golang.org/cl/127875 mentions this issue: |
See also my last comment in #2559 about using go generate for some of this. |
Change https://golang.org/cl/119715 mentions this issue: |
…cating Updates #26775 Change-Id: I83c9eeda59769d2f35e0cc98f3a8579861d5978b Reviewed-on: https://go-review.googlesource.com/119715 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
Updates golang/go#26775 Change-Id: Iea95ea07bb0fed42410efb4e8420d8e9a17704fe Reviewed-on: https://go-review.googlesource.com/127664 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Saves 22KB of memory in stdlib packages. Updates #26775 Change-Id: Ia19fe7aff61f6e2ddd83cd35969d7ff94526591f Reviewed-on: https://go-review.googlesource.com/127661 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Compile go/doc's 4 regexps lazily, on demand. Also, add a test for the one that had no test coverage. This reduces init-time CPU as well as heap by ~20KB when they're not used, which seems to be common enough. As an example, cmd/doc only seems to use 1 of them. (as noted by temporary print statements) Updates #26775 Change-Id: I85df89b836327a53fb8e1ace3f92480374270368 Reviewed-on: https://go-review.googlesource.com/127875 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Don't open files or do sysctls in init. Updates #26775 Change-Id: I017bed6c24ef1e4bc30040120349fb779f203225 Reviewed-on: https://go-review.googlesource.com/127655 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Saves 36KB of memory in stdlib packages. Updates #26775 Change-Id: I0f9d7b17d9768f6fb980d5fbba7c45920215a5fc Reviewed-on: https://go-review.googlesource.com/127735 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Using all.go that Brad posted in the OP, and using
|
Those top two |
Saves 6KB of memory in stdlib packages. Updates #26775 Change-Id: I1a6184cefa78e9a3c034fa84506fdfe0fec27add Reviewed-on: https://go-review.googlesource.com/127736 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Change https://golang.org/cl/166459 mentions this issue: |
The first biggest offender was crypto/des.init at ~1%. It's cryptographically broken and the init function is relatively expensive, which is unfortunate as both crypto/tls and crypto/x509 (and by extension, cmd/go) import it. Hide the work behind sync.Once. The second biggest offender was flag.sortFlags at just under 1%, used by the Visit flagset methods. It allocated two slices, which made a difference as cmd/go iterates over multiple flagsets during init. Use a single slice with a direct sort.Interface implementation. Another big offender is initializing global maps. Reducing this work in cmd/go/internal/imports and net/textproto gives us close to another whole 1% in saved work. The former can use map literals, and the latter can hide the work behind sync.Once. Finally, compress/flate used newHuffmanBitWriter as part of init, which allocates many objects and slices. Yet it only used one of the slice fields. Allocating just that slice saves a surprising ~0.3%, since we generated a lot of unnecessary garbage. All in all, these little pieces amount to just over 3% saved CPU time. name old time/op new time/op delta ExecGoEnv-8 3.61ms ± 1% 3.50ms ± 0% -3.02% (p=0.000 n=10+10) Updates #26775. Updates #29382. Change-Id: I915416e88a874c63235ba512617c8aef35c0ca8b Reviewed-on: https://go-review.googlesource.com/c/go/+/166459 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Change https://golang.org/cl/170317 mentions this issue: |
Change https://golang.org/cl/210284 mentions this issue: |
…nation Fixes #36021 Updates #2559 Updates #26775 Change-Id: I2e6708691311035b63866f25d5b4b3977a118290 Reviewed-on: https://go-review.googlesource.com/c/go/+/210284 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Rob Pike <r@golang.org>
…nation Fixes golang#36021 Updates golang#2559 Updates golang#26775 Change-Id: I2e6708691311035b63866f25d5b4b3977a118290 Reviewed-on: https://go-review.googlesource.com/c/go/+/210284 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Rob Pike <r@golang.org>
Isn't this resolved? ❯ go version
go version go1.17.2 darwin/amd64
❯ go build all.go
❯ GODEBUG=memprofilerate=1 ./all
289408
❯ go tool pprof /tmp/all.mem.prof
Type: inuse_space
Time: Dec 5, 2021 at 12:35pm (CET)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top 50
Showing nodes accounting for 9751.47kB, 100% of 9751.47kB total
flat flat% sum% cum cum%
3585.42kB 36.77% 36.77% 3585.42kB 36.77% runtime.malg
3092.09kB 31.71% 68.48% 3092.09kB 31.71% runtime.procresize
1537.69kB 15.77% 84.25% 4610.75kB 47.28% runtime.allocm
1024.25kB 10.50% 94.75% 2048.66kB 21.01% runtime.mcommoninit
512.02kB 5.25% 100% 512.02kB 5.25% text/template/parse.(*Tree).newList (inline)
0 0% 100% 512.02kB 5.25% html/template.(*Template).Parse
0 0% 100% 512.02kB 5.25% net/rpc.init
0 0% 100% 512.02kB 5.25% runtime.doInit
0 0% 100% 512.02kB 5.25% runtime.main
0 0% 100% 2049.09kB 21.01% runtime.main.func1
0 0% 100% 1024.41kB 10.51% runtime.mpreinit
0 0% 100% 3073.86kB 31.52% runtime.mstart
0 0% 100% 3073.86kB 31.52% runtime.mstart0
0 0% 100% 3073.86kB 31.52% runtime.mstart1
0 0% 100% 2561.30kB 26.27% runtime.mstartm0
0 0% 100% 2561.30kB 26.27% runtime.newextram
0 0% 100% 2561.66kB 26.27% runtime.newm
0 0% 100% 512.20kB 5.25% runtime.newproc
0 0% 100% 1024.41kB 10.51% runtime.newproc.func1
0 0% 100% 1024.41kB 10.51% runtime.newproc1
0 0% 100% 2561.30kB 26.27% runtime.oneNewExtraM
0 0% 100% 512.56kB 5.26% runtime.resetspinning
0 0% 100% 3604.29kB 36.96% runtime.rt0_go
0 0% 100% 3092.09kB 31.71% runtime.schedinit
0 0% 100% 512.56kB 5.26% runtime.schedule
0 0% 100% 512.56kB 5.26% runtime.startm
0 0% 100% 2561.30kB 26.27% runtime.systemstack
0 0% 100% 512.56kB 5.26% runtime.wakep
0 0% 100% 512.02kB 5.25% text/template.(*Template).Parse
0 0% 100% 512.02kB 5.25% text/template/parse.(*Tree).Parse
0 0% 100% 512.02kB 5.25% text/template/parse.(*Tree).action
0 0% 100% 512.02kB 5.25% text/template/parse.(*Tree).itemList
0 0% 100% 512.02kB 5.25% text/template/parse.(*Tree).parse
0 0% 100% 512.02kB 5.25% text/template/parse.(*Tree).parseControl
0 0% 100% 512.02kB 5.25% text/template/parse.(*Tree).rangeControl
0 0% 100% 512.02kB 5.25% text/template/parse.(*Tree).textOrAction
0 0% 100% 512.02kB 5.25% text/template/parse.Parse
(pprof) |
Updates golang/go#26775 Change-Id: Iea95ea07bb0fed42410efb4e8420d8e9a17704fe Reviewed-on: https://go-review.googlesource.com/127664 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Updates golang/go#26775 Change-Id: Iea95ea07bb0fed42410efb4e8420d8e9a17704fe Reviewed-on: https://go-review.googlesource.com/127664 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Tracking bug to reduce init-time CPU & memory usage.
Previously:
https://go-review.googlesource.com/c/go/+/127075 - html: lazily populate Unescape tables
https://go-review.googlesource.com/c/net/+/127275 - http2/hpack: lazily build huffman table on first use
Open:
#26752 - x/text/unicode/norm: reduce init-time memory usage
The program https://play.golang.org/p/9ervXCWzV_z is useful to find offenders:
The text was updated successfully, but these errors were encountered: