Tracking bug to reduce init-time CPU & memory usage.
Previously:
https://go-review.googlesource.com/c/go/+/127075 - html: lazily populate Unescape tables
https://go-review.googlesource.com/c/net/+/127275 - http2/hpack: lazily build huffman table on first use
Open:
#26752 - x/text/unicode/norm: reduce init-time memory usage
The program https://play.golang.org/p/9ervXCWzV_z is useful to find offenders:
$ go build all.go
$ GODEBUG=memprofilerate=1 ./all
346768
$ go tool pprof /tmp/all.mem.prof
(pprof) top 50
Showing nodes accounting for 286.24kB, 88.87% of 322.09kB total
Dropped 228 nodes (cum <= 1.61kB)
Showing top 50 nodes out of 133
flat flat% sum% cum cum%
37.03kB 11.50% 11.50% 37.03kB 11.50% runtime.procresize
32kB 9.94% 21.43% 32kB 9.94% hash/crc64.makeSlicingBy8Table
21.39kB 6.64% 28.07% 21.39kB 6.64% vendor/golang_org/x/text/unicode/norm.init
19.98kB 6.20% 34.28% 19.98kB 6.20% strings.NewReplacer
18.95kB 5.88% 40.16% 18.97kB 5.89% encoding/xml.init
12.17kB 3.78% 43.94% 12.17kB 3.78% unicode.init
10.92kB 3.39% 47.33% 10.92kB 3.39% regexp/syntax.appendRange
10.61kB 3.29% 50.63% 10.61kB 3.29% vendor/golang_org/x/net/http2/hpack.(*headerFieldTable).addEntry (inline)
10.38kB 3.22% 53.85% 12.62kB 3.92% sync.(*Map).LoadOrStore
9.38kB 2.91% 56.76% 9.38kB 2.91% runtime.malg
9.17kB 2.85% 59.61% 9.22kB 2.86% html/template.init
8.70kB 2.70% 62.31% 69.47kB 21.57% net/http.init
6.84kB 2.12% 64.43% 6.84kB 2.12% regexp/syntax.(*compiler).inst (inline)
6.12kB 1.90% 66.34% 13.12kB 4.07% runtime.allocm
5.25kB 1.63% 67.97% 5.25kB 1.63% math/rand.NewSource
5.06kB 1.57% 69.54% 5.47kB 1.70% net/http.init.0
4.03kB 1.25% 70.79% 4.22kB 1.31% net/http.init.2
3.94kB 1.22% 72.01% 3.95kB 1.23% debug/dwarf.init
3.72kB 1.15% 73.17% 3.72kB 1.15% errors.New
3.62kB 1.13% 74.29% 12.13kB 3.77% encoding/gob.validUserType
3.62kB 1.13% 75.42% 3.62kB 1.13% regexp/syntax.init
2.91kB 0.9% 76.32% 2.91kB 0.9% go/types.(*Scope).Insert
2.70kB 0.84% 77.16% 26.97kB 8.37% encoding/gob.init
2.69kB 0.83% 77.99% 2.69kB 0.83% flag.(*FlagSet).Var
2.59kB 0.81% 78.80% 2.59kB 0.81% net/mail.init.0
2.25kB 0.7% 79.50% 2.25kB 0.7% compress/flate.(*huffmanEncoder).generate
2.25kB 0.7% 80.20% 2.25kB 0.7% net/textproto.init.0
2.09kB 0.65% 80.85% 2.09kB 0.65% image/jpeg.(*huffmanLUT).init
1.89kB 0.59% 81.43% 1.89kB 0.59% compress/flate.newHuffmanEncoder
1.80kB 0.56% 81.99% 1.80kB 0.56% go/types.NewTypeName
1.77kB 0.55% 82.54% 1.77kB 0.55% text/template/parse.(*Tree).newText (inline)
1.75kB 0.54% 83.08% 4.38kB 1.36% runtime.mcommoninit
1.66kB 0.51% 83.60% 8.84kB 2.74% go/types.init
1.64kB 0.51% 84.11% 28.67kB 8.90% go/doc.init
1.59kB 0.49% 84.60% 3.57kB 1.11% encoding/gob.bootstrapType
1.55kB 0.48% 85.08% 2.98kB 0.93% net.init
1.39kB 0.43% 85.51% 32.34kB 10.04% go/build.init
1.17kB 0.36% 85.88% 3.09kB 0.96% text/template.init
1.16kB 0.36% 86.24% 3.55kB 1.10% encoding/gob.newTypeObject
1.06kB 0.33% 86.57% 14.55kB 4.52% encoding/gob.RegisterName
1.02kB 0.32% 86.88% 4.57kB 1.42% encoding/gob.buildTypeInfo
0.94kB 0.29% 87.17% 21.06kB 6.54% regexp.compile
0.91kB 0.28% 87.45% 16.88kB 5.24% encoding/gob.init.1
0.75kB 0.23% 87.69% 1.84kB 0.57% html/template.New
0.69kB 0.21% 87.90% 2.13kB 0.66% reflect.(*rtype).ptrTo
0.67kB 0.21% 88.11% 7.20kB 2.24% crypto/tls.init
0.67kB 0.21% 88.32% 22.42kB 6.96% vendor/golang_org/x/net/http/httpguts.init
0.61kB 0.19% 88.51% 14.49kB 4.50% archive/tar.init
0.61kB 0.19% 88.70% 5.40kB 1.68% net/http/pprof.init
0.56kB 0.17% 88.87% 7.81kB 2.43% runtime.systemstack
(pprof) top --cum 50
Showing nodes accounting for 222.77kB, 69.16% of 322.09kB total
Dropped 228 nodes (cum <= 1.61kB)
Showing top 50 nodes out of 133
flat flat% sum% cum cum%
0.09kB 0.029% 0.029% 268.74kB 83.44% runtime.main
0 0% 0.029% 266.68kB 82.80% main.init
0 0% 0.029% 70.38kB 21.85% expvar.init
8.70kB 2.70% 2.73% 69.47kB 21.57% net/http.init
0 0% 2.73% 37.41kB 11.61% runtime.rt0_go
37.03kB 11.50% 14.23% 37.03kB 11.50% runtime.procresize
0 0% 14.23% 37.03kB 11.50% runtime.schedinit
1.39kB 0.43% 14.66% 32.34kB 10.04% go/build.init
0 0% 14.66% 32kB 9.94% hash/crc64.init
32kB 9.94% 24.60% 32kB 9.94% hash/crc64.makeSlicingBy8Table
1.64kB 0.51% 25.10% 28.67kB 8.90% go/doc.init
2.70kB 0.84% 25.94% 26.97kB 8.37% encoding/gob.init
0.67kB 0.21% 26.15% 22.42kB 6.96% vendor/golang_org/x/net/http/httpguts.init
0 0% 26.15% 21.75kB 6.75% vendor/golang_org/x/net/idna.init
21.39kB 6.64% 32.79% 21.39kB 6.64% vendor/golang_org/x/text/unicode/norm.init
0 0% 32.79% 21.06kB 6.54% regexp.Compile
0 0% 32.79% 21.06kB 6.54% regexp.MustCompile
0.94kB 0.29% 33.09% 21.06kB 6.54% regexp.compile
19.98kB 6.20% 39.29% 19.98kB 6.20% strings.NewReplacer
18.95kB 5.88% 45.17% 18.97kB 5.89% encoding/xml.init
0.91kB 0.28% 45.46% 16.88kB 5.24% encoding/gob.init.1
0 0% 45.46% 14.55kB 4.52% encoding/gob.Register
1.06kB 0.33% 45.79% 14.55kB 4.52% encoding/gob.RegisterName
0 0% 45.79% 14.55kB 4.52% encoding/gob.registerBasics
0.61kB 0.19% 45.97% 14.49kB 4.50% archive/tar.init
0 0% 45.97% 13.70kB 4.25% fmt.init
6.12kB 1.90% 47.88% 13.12kB 4.07% runtime.allocm
0 0% 47.88% 12.72kB 3.95% regexp/syntax.Parse
10.38kB 3.22% 51.10% 12.62kB 3.92% sync.(*Map).LoadOrStore
0 0% 51.10% 12.17kB 3.78% reflect.init
12.17kB 3.78% 54.88% 12.17kB 3.78% unicode.init
0 0% 54.88% 12.13kB 3.77% encoding/gob.userType
3.62kB 1.13% 56.00% 12.13kB 3.77% encoding/gob.validUserType
0 0% 56.00% 11.94kB 3.71% runtime.mstart
0 0% 56.00% 11.50kB 3.57% regexp/syntax.(*parser).parseClass
0 0% 56.00% 11.25kB 3.49% runtime.newm
10.92kB 3.39% 59.39% 10.92kB 3.39% regexp/syntax.appendRange
0 0% 59.39% 10.81kB 3.36% vendor/golang_org/x/net/http2/hpack.init
0.05kB 0.015% 59.41% 10.75kB 3.34% vendor/golang_org/x/net/http2/hpack.newStaticTable
10.61kB 3.29% 62.70% 10.61kB 3.29% vendor/golang_org/x/net/http2/hpack.(*headerFieldTable).addEntry (inline)
0 0% 62.70% 10.50kB 3.26% regexp/syntax.(*parser).parseUnicodeClass
0 0% 62.70% 10.50kB 3.26% regexp/syntax.appendTable
9.38kB 2.91% 65.61% 9.38kB 2.91% runtime.malg
9.17kB 2.85% 68.46% 9.22kB 2.86% html/template.init
0 0% 68.46% 9.05kB 2.81% go/importer.init
0 0% 68.46% 9.05kB 2.81% go/internal/gccgoimporter.init
1.66kB 0.51% 68.97% 8.84kB 2.74% go/types.init
0.05kB 0.015% 68.99% 8.80kB 2.73% mime/multipart.init
0.56kB 0.17% 69.16% 7.81kB 2.43% runtime.systemstack
0 0% 69.16% 7.69kB 2.39% html/template.(*Template).Parse
Tracking bug to reduce init-time CPU & memory usage.
Previously:
https://go-review.googlesource.com/c/go/+/127075 - html: lazily populate Unescape tables
https://go-review.googlesource.com/c/net/+/127275 - http2/hpack: lazily build huffman table on first use
Open:
#26752 - x/text/unicode/norm: reduce init-time memory usage
The program https://play.golang.org/p/9ervXCWzV_z is useful to find offenders: