Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

all: reduce init-time CPU & memory usage #26775

Open
bradfitz opened this issue Aug 2, 2018 · 14 comments

Comments

@bradfitz
Copy link
Member

@bradfitz bradfitz commented Aug 2, 2018

Tracking bug to reduce init-time CPU & memory usage.

Previously:

https://go-review.googlesource.com/c/go/+/127075 - html: lazily populate Unescape tables
https://go-review.googlesource.com/c/net/+/127275 - http2/hpack: lazily build huffman table on first use

Open:

#26752 - x/text/unicode/norm: reduce init-time memory usage

The program https://play.golang.org/p/9ervXCWzV_z is useful to find offenders:

$ go build all.go
$ GODEBUG=memprofilerate=1  ./all
346768
$ go tool pprof /tmp/all.mem.prof 
(pprof) top 50
Showing nodes accounting for 286.24kB, 88.87% of 322.09kB total
Dropped 228 nodes (cum <= 1.61kB)
Showing top 50 nodes out of 133
      flat  flat%   sum%        cum   cum%
   37.03kB 11.50% 11.50%    37.03kB 11.50%  runtime.procresize
      32kB  9.94% 21.43%       32kB  9.94%  hash/crc64.makeSlicingBy8Table
   21.39kB  6.64% 28.07%    21.39kB  6.64%  vendor/golang_org/x/text/unicode/norm.init
   19.98kB  6.20% 34.28%    19.98kB  6.20%  strings.NewReplacer
   18.95kB  5.88% 40.16%    18.97kB  5.89%  encoding/xml.init
   12.17kB  3.78% 43.94%    12.17kB  3.78%  unicode.init
   10.92kB  3.39% 47.33%    10.92kB  3.39%  regexp/syntax.appendRange
   10.61kB  3.29% 50.63%    10.61kB  3.29%  vendor/golang_org/x/net/http2/hpack.(*headerFieldTable).addEntry (inline)
   10.38kB  3.22% 53.85%    12.62kB  3.92%  sync.(*Map).LoadOrStore
    9.38kB  2.91% 56.76%     9.38kB  2.91%  runtime.malg
    9.17kB  2.85% 59.61%     9.22kB  2.86%  html/template.init
    8.70kB  2.70% 62.31%    69.47kB 21.57%  net/http.init
    6.84kB  2.12% 64.43%     6.84kB  2.12%  regexp/syntax.(*compiler).inst (inline)
    6.12kB  1.90% 66.34%    13.12kB  4.07%  runtime.allocm
    5.25kB  1.63% 67.97%     5.25kB  1.63%  math/rand.NewSource
    5.06kB  1.57% 69.54%     5.47kB  1.70%  net/http.init.0
    4.03kB  1.25% 70.79%     4.22kB  1.31%  net/http.init.2
    3.94kB  1.22% 72.01%     3.95kB  1.23%  debug/dwarf.init
    3.72kB  1.15% 73.17%     3.72kB  1.15%  errors.New
    3.62kB  1.13% 74.29%    12.13kB  3.77%  encoding/gob.validUserType
    3.62kB  1.13% 75.42%     3.62kB  1.13%  regexp/syntax.init
    2.91kB   0.9% 76.32%     2.91kB   0.9%  go/types.(*Scope).Insert
    2.70kB  0.84% 77.16%    26.97kB  8.37%  encoding/gob.init
    2.69kB  0.83% 77.99%     2.69kB  0.83%  flag.(*FlagSet).Var
    2.59kB  0.81% 78.80%     2.59kB  0.81%  net/mail.init.0
    2.25kB   0.7% 79.50%     2.25kB   0.7%  compress/flate.(*huffmanEncoder).generate
    2.25kB   0.7% 80.20%     2.25kB   0.7%  net/textproto.init.0
    2.09kB  0.65% 80.85%     2.09kB  0.65%  image/jpeg.(*huffmanLUT).init
    1.89kB  0.59% 81.43%     1.89kB  0.59%  compress/flate.newHuffmanEncoder
    1.80kB  0.56% 81.99%     1.80kB  0.56%  go/types.NewTypeName
    1.77kB  0.55% 82.54%     1.77kB  0.55%  text/template/parse.(*Tree).newText (inline)
    1.75kB  0.54% 83.08%     4.38kB  1.36%  runtime.mcommoninit
    1.66kB  0.51% 83.60%     8.84kB  2.74%  go/types.init
    1.64kB  0.51% 84.11%    28.67kB  8.90%  go/doc.init
    1.59kB  0.49% 84.60%     3.57kB  1.11%  encoding/gob.bootstrapType
    1.55kB  0.48% 85.08%     2.98kB  0.93%  net.init
    1.39kB  0.43% 85.51%    32.34kB 10.04%  go/build.init
    1.17kB  0.36% 85.88%     3.09kB  0.96%  text/template.init
    1.16kB  0.36% 86.24%     3.55kB  1.10%  encoding/gob.newTypeObject
    1.06kB  0.33% 86.57%    14.55kB  4.52%  encoding/gob.RegisterName
    1.02kB  0.32% 86.88%     4.57kB  1.42%  encoding/gob.buildTypeInfo
    0.94kB  0.29% 87.17%    21.06kB  6.54%  regexp.compile
    0.91kB  0.28% 87.45%    16.88kB  5.24%  encoding/gob.init.1
    0.75kB  0.23% 87.69%     1.84kB  0.57%  html/template.New
    0.69kB  0.21% 87.90%     2.13kB  0.66%  reflect.(*rtype).ptrTo
    0.67kB  0.21% 88.11%     7.20kB  2.24%  crypto/tls.init
    0.67kB  0.21% 88.32%    22.42kB  6.96%  vendor/golang_org/x/net/http/httpguts.init
    0.61kB  0.19% 88.51%    14.49kB  4.50%  archive/tar.init
    0.61kB  0.19% 88.70%     5.40kB  1.68%  net/http/pprof.init
    0.56kB  0.17% 88.87%     7.81kB  2.43%  runtime.systemstack

(pprof) top --cum 50
Showing nodes accounting for 222.77kB, 69.16% of 322.09kB total
Dropped 228 nodes (cum <= 1.61kB)
Showing top 50 nodes out of 133
      flat  flat%   sum%        cum   cum%
    0.09kB 0.029% 0.029%   268.74kB 83.44%  runtime.main
         0     0% 0.029%   266.68kB 82.80%  main.init
         0     0% 0.029%    70.38kB 21.85%  expvar.init
    8.70kB  2.70%  2.73%    69.47kB 21.57%  net/http.init
         0     0%  2.73%    37.41kB 11.61%  runtime.rt0_go
   37.03kB 11.50% 14.23%    37.03kB 11.50%  runtime.procresize
         0     0% 14.23%    37.03kB 11.50%  runtime.schedinit
    1.39kB  0.43% 14.66%    32.34kB 10.04%  go/build.init
         0     0% 14.66%       32kB  9.94%  hash/crc64.init
      32kB  9.94% 24.60%       32kB  9.94%  hash/crc64.makeSlicingBy8Table
    1.64kB  0.51% 25.10%    28.67kB  8.90%  go/doc.init
    2.70kB  0.84% 25.94%    26.97kB  8.37%  encoding/gob.init
    0.67kB  0.21% 26.15%    22.42kB  6.96%  vendor/golang_org/x/net/http/httpguts.init
         0     0% 26.15%    21.75kB  6.75%  vendor/golang_org/x/net/idna.init
   21.39kB  6.64% 32.79%    21.39kB  6.64%  vendor/golang_org/x/text/unicode/norm.init
         0     0% 32.79%    21.06kB  6.54%  regexp.Compile
         0     0% 32.79%    21.06kB  6.54%  regexp.MustCompile
    0.94kB  0.29% 33.09%    21.06kB  6.54%  regexp.compile
   19.98kB  6.20% 39.29%    19.98kB  6.20%  strings.NewReplacer
   18.95kB  5.88% 45.17%    18.97kB  5.89%  encoding/xml.init
    0.91kB  0.28% 45.46%    16.88kB  5.24%  encoding/gob.init.1
         0     0% 45.46%    14.55kB  4.52%  encoding/gob.Register
    1.06kB  0.33% 45.79%    14.55kB  4.52%  encoding/gob.RegisterName
         0     0% 45.79%    14.55kB  4.52%  encoding/gob.registerBasics
    0.61kB  0.19% 45.97%    14.49kB  4.50%  archive/tar.init
         0     0% 45.97%    13.70kB  4.25%  fmt.init
    6.12kB  1.90% 47.88%    13.12kB  4.07%  runtime.allocm
         0     0% 47.88%    12.72kB  3.95%  regexp/syntax.Parse
   10.38kB  3.22% 51.10%    12.62kB  3.92%  sync.(*Map).LoadOrStore
         0     0% 51.10%    12.17kB  3.78%  reflect.init
   12.17kB  3.78% 54.88%    12.17kB  3.78%  unicode.init
         0     0% 54.88%    12.13kB  3.77%  encoding/gob.userType
    3.62kB  1.13% 56.00%    12.13kB  3.77%  encoding/gob.validUserType
         0     0% 56.00%    11.94kB  3.71%  runtime.mstart
         0     0% 56.00%    11.50kB  3.57%  regexp/syntax.(*parser).parseClass
         0     0% 56.00%    11.25kB  3.49%  runtime.newm
   10.92kB  3.39% 59.39%    10.92kB  3.39%  regexp/syntax.appendRange
         0     0% 59.39%    10.81kB  3.36%  vendor/golang_org/x/net/http2/hpack.init
    0.05kB 0.015% 59.41%    10.75kB  3.34%  vendor/golang_org/x/net/http2/hpack.newStaticTable
   10.61kB  3.29% 62.70%    10.61kB  3.29%  vendor/golang_org/x/net/http2/hpack.(*headerFieldTable).addEntry (inline)
         0     0% 62.70%    10.50kB  3.26%  regexp/syntax.(*parser).parseUnicodeClass
         0     0% 62.70%    10.50kB  3.26%  regexp/syntax.appendTable
    9.38kB  2.91% 65.61%     9.38kB  2.91%  runtime.malg
    9.17kB  2.85% 68.46%     9.22kB  2.86%  html/template.init
         0     0% 68.46%     9.05kB  2.81%  go/importer.init
         0     0% 68.46%     9.05kB  2.81%  go/internal/gccgoimporter.init
    1.66kB  0.51% 68.97%     8.84kB  2.74%  go/types.init
    0.05kB 0.015% 68.99%     8.80kB  2.73%  mime/multipart.init
    0.56kB  0.17% 69.16%     7.81kB  2.43%  runtime.systemstack
         0     0% 69.16%     7.69kB  2.39%  html/template.(*Template).Parse
@bradfitz bradfitz added the Performance label Aug 2, 2018
@bradfitz bradfitz added this to the Unplanned milestone Aug 2, 2018
@bradfitz bradfitz added the help wanted label Aug 2, 2018
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Aug 2, 2018

Change https://golang.org/cl/127655 mentions this issue: net: lazily look up the listenerBacklog value on first use

zegl added a commit to zegl/go that referenced this issue Aug 2, 2018
Defer call to registerBasics until the first invocation of NewDecoder() or NewEncoder()

This saves ~13.5kB of allocation in init()

Updates golang#26775
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Aug 2, 2018

Change https://golang.org/cl/127660 mentions this issue: encoding/gob: lazily initialize registerBasics

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Aug 2, 2018

Change https://golang.org/cl/127661 mentions this issue: strings: select Replacer algorithm and build machine lazily

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Aug 2, 2018

Change https://golang.org/cl/127664 mentions this issue: http2: reduce init-time work & allocations

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Aug 3, 2018

Change https://golang.org/cl/127735 mentions this issue: hash/crc64: lazily initialize slice8Tables

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Aug 3, 2018

Change https://golang.org/cl/127736 mentions this issue: net/mail: lazily initialize dateLayouts

@ysmolsky

This comment has been minimized.

Copy link
Member

@ysmolsky ysmolsky commented Aug 3, 2018

Wow, that was picked up real fast by the others!

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Aug 3, 2018

Change https://golang.org/cl/127875 mentions this issue: go/doc: compile regexps lazily

@josharian

This comment has been minimized.

Copy link
Contributor

@josharian josharian commented Aug 5, 2018

See also my last comment in #2559 about using go generate for some of this.

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Aug 21, 2018

Change https://golang.org/cl/119715 mentions this issue: encoding/json: change reflect.Type usage to be more idiomatic

gopherbot pushed a commit that referenced this issue Aug 21, 2018
…cating

Updates #26775

Change-Id: I83c9eeda59769d2f35e0cc98f3a8579861d5978b
Reviewed-on: https://go-review.googlesource.com/119715
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
gopherbot pushed a commit to golang/net that referenced this issue Aug 21, 2018
Updates golang/go#26775

Change-Id: Iea95ea07bb0fed42410efb4e8420d8e9a17704fe
Reviewed-on: https://go-review.googlesource.com/127664
Reviewed-by: Ian Lance Taylor <iant@golang.org>
gopherbot pushed a commit that referenced this issue Aug 21, 2018
Saves 22KB of memory in stdlib packages.

Updates #26775

Change-Id: Ia19fe7aff61f6e2ddd83cd35969d7ff94526591f
Reviewed-on: https://go-review.googlesource.com/127661
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
gopherbot pushed a commit that referenced this issue Aug 21, 2018
Compile go/doc's 4 regexps lazily, on demand.

Also, add a test for the one that had no test coverage.

This reduces init-time CPU as well as heap by ~20KB when they're not
used, which seems to be common enough. As an example, cmd/doc only
seems to use 1 of them. (as noted by temporary print statements)

Updates #26775

Change-Id: I85df89b836327a53fb8e1ace3f92480374270368
Reviewed-on: https://go-review.googlesource.com/127875
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
gopherbot pushed a commit that referenced this issue Aug 21, 2018
Don't open files or do sysctls in init.

Updates #26775

Change-Id: I017bed6c24ef1e4bc30040120349fb779f203225
Reviewed-on: https://go-review.googlesource.com/127655
Reviewed-by: Ian Lance Taylor <iant@golang.org>
gopherbot pushed a commit that referenced this issue Aug 21, 2018
Saves 36KB of memory in stdlib packages.

Updates #26775

Change-Id: I0f9d7b17d9768f6fb980d5fbba7c45920215a5fc
Reviewed-on: https://go-review.googlesource.com/127735
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@mark-rushakoff

This comment has been minimized.

Copy link
Contributor

@mark-rushakoff mark-rushakoff commented Aug 21, 2018

Using all.go that Brad posted in the OP, and using go version devel +aa311fecda Tue Aug 21 22:11:05 2018 +0000 darwin/amd64, here are the current results:

$ gotip build all.go
$ GODEBUG=memprofilerate=1  ./all
278848
$ gotip tool pprof /tmp/all.mem.prof
Type: inuse_space
Time: Aug 21, 2018 at 3:15pm (PDT)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top 50
Showing nodes accounting for 228.01kB, 87.93% of 259.32kB total
Dropped 218 nodes (cum <= 1.30kB)
Showing top 50 nodes out of 139
      flat  flat%   sum%        cum   cum%
   37.03kB 14.28% 14.28%    37.03kB 14.28%  runtime.procresize
   21.39kB  8.25% 22.53%    21.39kB  8.25%  vendor/golang_org/x/text/unicode/norm.init
   18.95kB  7.31% 29.84%    18.97kB  7.31%  encoding/xml.init
   12.17kB  4.69% 34.53%    12.17kB  4.69%  unicode.init
   10.81kB  4.17% 38.70%    10.81kB  4.17%  vendor/golang_org/x/net/http2/hpack.(*headerFieldTable).addEntry (inline)
   10.58kB  4.08% 42.78%    12.82kB  4.94%  sync.(*Map).LoadOrStore
    9.17kB  3.54% 46.32%     9.22kB  3.55%  html/template.init
       9kB  3.47% 49.79%        9kB  3.47%  runtime.malg
    8.70kB  3.36% 53.14%    56.52kB 21.79%  net/http.init
       7kB  2.70% 55.84%       14kB  5.40%  runtime.allocm
    6.03kB  2.33% 58.17%     6.03kB  2.33%  syscall.copyenv
    5.25kB  2.02% 60.19%     5.25kB  2.02%  math/rand.NewSource
    4.78kB  1.84% 62.04%     5.19kB  2.00%  net/http.init.0
    4.03kB  1.55% 63.59%     4.22kB  1.63%  net/http.init.2
    3.94kB  1.52% 65.11%     3.95kB  1.52%  debug/dwarf.init
    3.81kB  1.47% 66.58%     3.81kB  1.47%  errors.New
    3.62kB  1.40% 67.98%    12.34kB  4.76%  encoding/gob.validUserType
    3.62kB  1.40% 69.38%     3.62kB  1.40%  regexp/syntax.init
    2.91kB  1.12% 70.50%     2.91kB  1.12%  go/types.(*Scope).Insert
    2.70kB  1.04% 71.54%    27.16kB 10.47%  encoding/gob.init
    2.69kB  1.04% 72.58%     2.69kB  1.04%  flag.(*FlagSet).Var
    2.59kB  1.00% 73.58%     2.59kB  1.00%  net/mail.init.0
    2.25kB  0.87% 74.44%     2.25kB  0.87%  compress/flate.(*huffmanEncoder).generate
    2.25kB  0.87% 75.31%     2.25kB  0.87%  net/textproto.init.0
    2.09kB  0.81% 76.12%     2.09kB  0.81%  image/jpeg.(*huffmanLUT).init
    1.89kB  0.73% 76.85%     1.89kB  0.73%  compress/flate.newHuffmanEncoder
    1.80kB  0.69% 77.54%     1.80kB  0.69%  go/types.NewTypeName
    1.77kB  0.68% 78.22%     1.77kB  0.68%  text/template/parse.(*Tree).newText
    1.75kB  0.67% 78.90%     4.38kB  1.69%  runtime.mcommoninit
    1.66kB  0.64% 79.53%     8.85kB  3.41%  go/types.init
    1.64kB  0.63% 80.17%    10.17kB  3.92%  go/doc.init
    1.59kB  0.61% 80.78%     3.74kB  1.44%  encoding/gob.bootstrapType
    1.56kB   0.6% 81.38%     1.56kB   0.6%  regexp/syntax.(*compiler).inst (inline)
    1.55kB   0.6% 81.98%     4.02kB  1.55%  net.init
    1.41kB  0.54% 82.52%     1.41kB  0.54%  go/types.newBuiltin (inline)
    1.39kB  0.54% 83.06%    13.84kB  5.34%  go/build.init
    1.38kB  0.53% 83.59%     1.38kB  0.53%  text/template.addValueFuncs
    1.31kB  0.51% 84.10%     1.31kB  0.51%  text/template/parse.(*Tree).newPipeline
    1.17kB  0.45% 84.55%     3.09kB  1.19%  text/template.init
    1.16kB  0.45% 85.00%     3.74kB  1.44%  encoding/gob.newTypeObject
    1.06kB  0.41% 85.41%    14.55kB  5.61%  encoding/gob.RegisterName
    1.02kB  0.39% 85.80%     4.76kB  1.83%  encoding/gob.buildTypeInfo
    0.75kB  0.29% 86.09%     1.84kB  0.71%  html/template.New
    0.75kB  0.29% 86.38%     7.88kB  3.04%  runtime.systemstack
    0.73kB  0.28% 86.66%    16.71kB  6.44%  encoding/gob.init.1
    0.69kB  0.27% 86.92%     2.13kB  0.82%  reflect.(*rtype).ptrTo
    0.67kB  0.26% 87.18%     6.92kB  2.67%  crypto/tls.init
    0.67kB  0.26% 87.44%    22.42kB  8.65%  vendor/golang_org/x/net/http/httpguts.init
    0.64kB  0.25% 87.69%     1.42kB  0.55%  net/http/cgi.init
    0.61kB  0.23% 87.93%    20.99kB  8.10%  archive/tar.init
(pprof) top --cum 50
Showing nodes accounting for 160.47kB, 61.88% of 259.32kB total
Dropped 218 nodes (cum <= 1.30kB)
Showing top 50 nodes out of 139
      flat  flat%   sum%        cum   cum%
    0.09kB 0.036% 0.036%   205.41kB 79.21%  runtime.main
         0     0% 0.036%   203.23kB 78.37%  main.init
         0     0% 0.036%    57.42kB 22.14%  expvar.init
    8.70kB  3.36%  3.39%    56.52kB 21.79%  net/http.init
         0     0%  3.39%    37.41kB 14.42%  runtime.rt0_go
   37.03kB 14.28% 17.67%    37.03kB 14.28%  runtime.procresize
         0     0% 17.67%    37.03kB 14.28%  runtime.schedinit
    2.70kB  1.04% 18.71%    27.16kB 10.47%  encoding/gob.init
    0.67kB  0.26% 18.97%    22.42kB  8.65%  vendor/golang_org/x/net/http/httpguts.init
         0     0% 18.97%    21.75kB  8.39%  vendor/golang_org/x/net/idna.init
   21.39kB  8.25% 27.22%    21.39kB  8.25%  vendor/golang_org/x/text/unicode/norm.init
    0.61kB  0.23% 27.46%    20.99kB  8.10%  archive/tar.init
         0     0% 27.46%    20.20kB  7.79%  fmt.init
   18.95kB  7.31% 34.77%    18.97kB  7.31%  encoding/xml.init
    0.73kB  0.28% 35.05%    16.71kB  6.44%  encoding/gob.init.1
         0     0% 35.05%    14.55kB  5.61%  encoding/gob.Register
    1.06kB  0.41% 35.46%    14.55kB  5.61%  encoding/gob.RegisterName
         0     0% 35.46%    14.55kB  5.61%  encoding/gob.registerBasics
       7kB  2.70% 38.16%       14kB  5.40%  runtime.allocm
    1.39kB  0.54% 38.69%    13.84kB  5.34%  go/build.init
   10.58kB  4.08% 42.77%    12.82kB  4.94%  sync.(*Map).LoadOrStore
         0     0% 42.77%    12.34kB  4.76%  encoding/gob.userType
    3.62kB  1.40% 44.17%    12.34kB  4.76%  encoding/gob.validUserType
         0     0% 44.17%    12.17kB  4.69%  reflect.init
   12.17kB  4.69% 48.87%    12.17kB  4.69%  unicode.init
         0     0% 48.87%       12kB  4.63%  runtime.newm
         0     0% 48.87%    11.02kB  4.25%  vendor/golang_org/x/net/http2/hpack.init
    0.05kB 0.018% 48.88%    10.95kB  4.22%  vendor/golang_org/x/net/http2/hpack.newStaticTable
   10.81kB  4.17% 53.05%    10.81kB  4.17%  vendor/golang_org/x/net/http2/hpack.(*headerFieldTable).addEntry (inline)
         0     0% 53.05%    10.25kB  3.95%  runtime.mstart
    1.64kB  0.63% 53.69%    10.17kB  3.92%  go/doc.init
    9.17kB  3.54% 57.22%     9.22kB  3.55%  html/template.init
         0     0% 57.22%     9.07kB  3.50%  go/importer.init
         0     0% 57.22%     9.07kB  3.50%  go/internal/gccgoimporter.init
       9kB  3.47% 60.69%        9kB  3.47%  runtime.malg
    1.66kB  0.64% 61.33%     8.85kB  3.41%  go/types.init
         0     0% 61.33%        8kB  3.08%  runtime.startm
    0.75kB  0.29% 61.62%     7.88kB  3.04%  runtime.systemstack
         0     0% 61.62%     7.82kB  3.02%  os.init
         0     0% 61.62%     7.80kB  3.01%  html/template.(*Template).Parse
         0     0% 61.62%     7.80kB  3.01%  text/template.(*Template).Parse
         0     0% 61.62%     7.39kB  2.85%  text/template/parse.Parse
         0     0% 61.62%     7.04kB  2.71%  go/types.init.0
    0.67kB  0.26% 61.88%     6.92kB  2.67%  crypto/tls.init
         0     0% 61.88%     6.89kB  2.66%  text/template/parse.(*Tree).Parse
         0     0% 61.88%     6.89kB  2.66%  text/template/parse.(*Tree).parse
         0     0% 61.88%     6.52kB  2.51%  text/template/parse.(*Tree).textOrAction
         0     0% 61.88%     6.16kB  2.37%  crypto/x509.init
         0     0% 61.88%     6.05kB  2.33%  os.Getwd
         0     0% 61.88%     6.03kB  2.33%  os.Getenv
@bradfitz

This comment has been minimized.

Copy link
Member Author

@bradfitz bradfitz commented Aug 21, 2018

Those top two vendor/golang_org/x ones fixed in x/ but need to be imported into std. That'll happen soon enough.

gopherbot pushed a commit that referenced this issue Aug 22, 2018
Saves 6KB of memory in stdlib packages.

Updates #26775

Change-Id: I1a6184cefa78e9a3c034fa84506fdfe0fec27add
Reviewed-on: https://go-review.googlesource.com/127736
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Mar 9, 2019

Change https://golang.org/cl/166459 mentions this issue: cmd/go: further reduce init work

gopherbot pushed a commit that referenced this issue Mar 31, 2019
The first biggest offender was crypto/des.init at ~1%. It's
cryptographically broken and the init function is relatively expensive,
which is unfortunate as both crypto/tls and crypto/x509 (and by
extension, cmd/go) import it. Hide the work behind sync.Once.

The second biggest offender was flag.sortFlags at just under 1%, used by
the Visit flagset methods. It allocated two slices, which made a
difference as cmd/go iterates over multiple flagsets during init.
Use a single slice with a direct sort.Interface implementation.

Another big offender is initializing global maps. Reducing this work in
cmd/go/internal/imports and net/textproto gives us close to another
whole 1% in saved work. The former can use map literals, and the latter
can hide the work behind sync.Once.

Finally, compress/flate used newHuffmanBitWriter as part of init, which
allocates many objects and slices. Yet it only used one of the slice
fields. Allocating just that slice saves a surprising ~0.3%, since we
generated a lot of unnecessary garbage.

All in all, these little pieces amount to just over 3% saved CPU time.

name         old time/op  new time/op  delta
ExecGoEnv-8  3.61ms ± 1%  3.50ms ± 0%  -3.02%  (p=0.000 n=10+10)

Updates #26775.
Updates #29382.

Change-Id: I915416e88a874c63235ba512617c8aef35c0ca8b
Reviewed-on: https://go-review.googlesource.com/c/go/+/166459
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Apr 1, 2019

Change https://golang.org/cl/170317 mentions this issue: net/textproto: simplify commonHeader initialization

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.