Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/compile, x/build/cmd/relui: update compiler's PGO profile as part of rc1 relui flow #60234

Open
cherrymui opened this issue May 16, 2023 · 8 comments
Open

cmd/compile, x/build/cmd/relui: update compiler's PGO profile as part of rc1 relui flow #60234

cherrymui opened this issue May 16, 2023 · 8 comments
Labels
Builders x/build issues (builders, bots, dashboards) NeedsFix The path to resolution is known, but the work has not been done. recurring Issues that should never be closed, but moved to the next milestone once fixed in the current one. release-blocker
Milestone
Go1.25

Comments

@cherrymui
Copy link
Member

CL https://golang.org/cl/451292 adds a default profile of the compiler, so we can build the compiler with PGO. As the compiler's source code evolves, we may want to periodically update the profile.

cc @prattmic @aclements
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label May 16, 2023
@cherrymui cherrymui added recurring Issues that should never be closed, but moved to the next milestone once fixed in the current one. and removed compiler/runtime Issues related to the Go compiler and/or runtime. labels May 16, 2023
@heschi heschi added the NeedsFix The path to resolution is known, but the work has not been done. label May 17, 2023
@aclements
Copy link
Member

We discussed this in the release team meeting today. The plan is to add this as a relui step, to be performed as part of creating rc1. We can also run the step as a one-off if necessary (e.g., after any major compiler refactoring).

Ideally we can shrink the profile a little. Currently it's ~350K, which is just big enough to be a little annoying to update regularly in git.

cc @golang/release

@aclements aclements changed the title cmd/compile: periodically update compiler's PGO profile cmd/compile: update compiler's PGO profile as part of rc1 relui flow May 23, 2023
@dmitshur dmitshur added the Builders x/build issues (builders, bots, dashboards) label May 23, 2023
@heschi
Copy link
Contributor

heschi commented May 23, 2023

We should discuss whether to try to get it done for this cycle. On the one hand it's not very necessary, but on the other it might be best to strike while the iron is hot.

@dmitshur dmitshur changed the title cmd/compile: update compiler's PGO profile as part of rc1 relui flow cmd/compile, x/build/cmd/relui: update compiler's PGO profile as part of rc1 relui flow May 25, 2023
@dmitshur dmitshur moved this to Planned in Go Release May 30, 2023
@cherrymui
Copy link
Member Author

Ideally we can shrink the profile a little

When I did it I put GOROOT in /tmp so the file paths are short. We could probably also trim cold nodes from the profile to reduce size.

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/546337 mentions this issue: cmd/compile: update PGO profile

gopherbot pushed a commit that referenced this issue Dec 4, 2023
@cherrymui
cmd/compile: update PGO profile
1926a0b 
Since the default.pgo profile is collected, there has been a lot
of development and refactoring in the compiler. It's time to
refresh the compiler's PGO profile. As we are in the freeze, there
will probably not be many major changes to the compiler, so this
should be good for Go 1.22.

Profile collected by running the cmd/compile/profile.sh script on
the linux-amd64-perf gomote.

Benchmark results on Linux/AMD64:

         │  nopgo.txt   │               old.txt               │              new.txt               │
         │    sec/op    │    sec/op     vs base               │   sec/op     vs base               │
Template    127.4m ± 0%    125.3m ± 0%  -1.65% (p=0.000 n=20)   123.5m ± 0%  -3.07% (p=0.000 n=20)
Unicode    105.94m ± 0%   100.77m ± 0%  -4.87% (p=0.000 n=20)   99.26m ± 0%  -6.30% (p=0.000 n=20)
GoTypes     798.2m ± 1%    784.4m ± 0%  -1.73% (p=0.000 n=20)   769.4m ± 1%  -3.61% (p=0.000 n=20)
Compiler    85.10m ± 0%    82.03m ± 1%  -3.61% (p=0.000 n=20)   79.77m ± 1%  -6.26% (p=0.000 n=20)
SSA          6.054 ± 0%     5.945 ± 0%  -1.81% (p=0.000 n=20)    5.811 ± 0%  -4.03% (p=0.000 n=20)
Flate       75.37m ± 1%    74.11m ± 0%  -1.66% (p=0.000 n=20)   73.02m ± 1%  -3.12% (p=0.000 n=20)
GoParser    144.6m ± 1%    141.7m ± 1%  -1.95% (p=0.000 n=20)   140.5m ± 1%  -2.83% (p=0.000 n=20)
Reflect     312.9m ± 1%    304.9m ± 1%  -2.56% (p=0.000 n=20)   300.4m ± 0%  -4.00% (p=0.000 n=20)
Tar         115.8m ± 0%    113.6m ± 0%  -1.88% (p=0.000 n=20)   111.9m ± 1%  -3.33% (p=0.000 n=20)
XML         145.9m ± 1%    143.6m ± 0%  -1.58% (p=0.000 n=20)   141.3m ± 1%  -3.15% (p=0.000 n=20)
geomean     224.1m         218.9m       -2.34%                  215.2m       -3.98%

This also shows that a stale profile (the old one) still brings a
speedup, but smaller, as expected.

As before, the profile is transferrable. Benchmark results on
Darwin/ARM64:

         │  nopgo.txt   │              old.txt               │               new.txt               │
         │    sec/op    │   sec/op     vs base               │    sec/op     vs base               │
Template   71.31m ± 34%   68.97m ± 1%  -3.28% (p=0.000 n=20)   68.66m ± 38%  -3.71% (p=0.030 n=20)
Unicode    58.70m ±  1%   55.64m ± 1%  -5.20% (p=0.000 n=20)   53.05m ±  1%  -9.61% (p=0.000 n=20)
GoTypes    422.9m ±  0%   412.7m ± 0%  -2.43% (p=0.000 n=20)   406.0m ±  1%  -4.01% (p=0.000 n=20)
Compiler   45.89m ±  1%   42.67m ± 2%  -7.00% (p=0.000 n=20)   42.96m ±  1%  -6.38% (p=0.000 n=20)
SSA         3.190 ±  0%    3.090 ± 0%  -3.15% (p=0.000 n=20)    3.007 ±  1%  -5.74% (p=0.000 n=20)
Flate      42.02m ±  1%   41.09m ± 1%  -2.20% (p=0.000 n=20)   40.58m ±  1%  -3.43% (p=0.000 n=20)
GoParser   75.75m ±  1%   73.84m ± 1%  -2.52% (p=0.000 n=20)   72.66m ±  1%  -4.08% (p=0.000 n=20)
Reflect    163.7m ±  1%   158.1m ± 1%  -3.39% (p=0.000 n=20)   155.3m ±  1%  -5.10% (p=0.000 n=20)
Tar        62.03m ±  1%   60.15m ± 0%  -3.02% (p=0.000 n=20)   59.74m ±  1%  -3.69% (p=0.000 n=20)
XML        79.38m ±  1%   77.32m ± 1%  -2.59% (p=0.000 n=20)   75.05m ±  2%  -5.45% (p=0.000 n=20)
geomean    120.9m         116.6m       -3.49%                  114.6m        -5.14%

For #60234.

Change-Id: I4ead6158f835c2e69c0f51ea24d044b82a7320cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/546337
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
ezz-no pushed a commit to ezz-no/go-ezzno that referenced this issue Feb 18, 2024
@cherrymui @ezz-no
cmd/compile: update PGO profile
620e968 
Since the default.pgo profile is collected, there has been a lot
of development and refactoring in the compiler. It's time to
refresh the compiler's PGO profile. As we are in the freeze, there
will probably not be many major changes to the compiler, so this
should be good for Go 1.22.

Profile collected by running the cmd/compile/profile.sh script on
the linux-amd64-perf gomote.

Benchmark results on Linux/AMD64:

         │  nopgo.txt   │               old.txt               │              new.txt               │
         │    sec/op    │    sec/op     vs base               │   sec/op     vs base               │
Template    127.4m ± 0%    125.3m ± 0%  -1.65% (p=0.000 n=20)   123.5m ± 0%  -3.07% (p=0.000 n=20)
Unicode    105.94m ± 0%   100.77m ± 0%  -4.87% (p=0.000 n=20)   99.26m ± 0%  -6.30% (p=0.000 n=20)
GoTypes     798.2m ± 1%    784.4m ± 0%  -1.73% (p=0.000 n=20)   769.4m ± 1%  -3.61% (p=0.000 n=20)
Compiler    85.10m ± 0%    82.03m ± 1%  -3.61% (p=0.000 n=20)   79.77m ± 1%  -6.26% (p=0.000 n=20)
SSA          6.054 ± 0%     5.945 ± 0%  -1.81% (p=0.000 n=20)    5.811 ± 0%  -4.03% (p=0.000 n=20)
Flate       75.37m ± 1%    74.11m ± 0%  -1.66% (p=0.000 n=20)   73.02m ± 1%  -3.12% (p=0.000 n=20)
GoParser    144.6m ± 1%    141.7m ± 1%  -1.95% (p=0.000 n=20)   140.5m ± 1%  -2.83% (p=0.000 n=20)
Reflect     312.9m ± 1%    304.9m ± 1%  -2.56% (p=0.000 n=20)   300.4m ± 0%  -4.00% (p=0.000 n=20)
Tar         115.8m ± 0%    113.6m ± 0%  -1.88% (p=0.000 n=20)   111.9m ± 1%  -3.33% (p=0.000 n=20)
XML         145.9m ± 1%    143.6m ± 0%  -1.58% (p=0.000 n=20)   141.3m ± 1%  -3.15% (p=0.000 n=20)
geomean     224.1m         218.9m       -2.34%                  215.2m       -3.98%

This also shows that a stale profile (the old one) still brings a
speedup, but smaller, as expected.

As before, the profile is transferrable. Benchmark results on
Darwin/ARM64:

         │  nopgo.txt   │              old.txt               │               new.txt               │
         │    sec/op    │   sec/op     vs base               │    sec/op     vs base               │
Template   71.31m ± 34%   68.97m ± 1%  -3.28% (p=0.000 n=20)   68.66m ± 38%  -3.71% (p=0.030 n=20)
Unicode    58.70m ±  1%   55.64m ± 1%  -5.20% (p=0.000 n=20)   53.05m ±  1%  -9.61% (p=0.000 n=20)
GoTypes    422.9m ±  0%   412.7m ± 0%  -2.43% (p=0.000 n=20)   406.0m ±  1%  -4.01% (p=0.000 n=20)
Compiler   45.89m ±  1%   42.67m ± 2%  -7.00% (p=0.000 n=20)   42.96m ±  1%  -6.38% (p=0.000 n=20)
SSA         3.190 ±  0%    3.090 ± 0%  -3.15% (p=0.000 n=20)    3.007 ±  1%  -5.74% (p=0.000 n=20)
Flate      42.02m ±  1%   41.09m ± 1%  -2.20% (p=0.000 n=20)   40.58m ±  1%  -3.43% (p=0.000 n=20)
GoParser   75.75m ±  1%   73.84m ± 1%  -2.52% (p=0.000 n=20)   72.66m ±  1%  -4.08% (p=0.000 n=20)
Reflect    163.7m ±  1%   158.1m ± 1%  -3.39% (p=0.000 n=20)   155.3m ±  1%  -5.10% (p=0.000 n=20)
Tar        62.03m ±  1%   60.15m ± 0%  -3.02% (p=0.000 n=20)   59.74m ±  1%  -3.69% (p=0.000 n=20)
XML        79.38m ±  1%   77.32m ± 1%  -2.59% (p=0.000 n=20)   75.05m ±  2%  -5.45% (p=0.000 n=20)
geomean    120.9m         116.6m       -3.49%                  114.6m        -5.14%

For golang#60234.

Change-Id: I4ead6158f835c2e69c0f51ea24d044b82a7320cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/546337
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/587195 mentions this issue: cmd/compile: update PGO profile

@cagedmantis cagedmantis added this to the Go1.23 milestone May 22, 2024
gopherbot pushed a commit that referenced this issue May 22, 2024
@cherrymui
cmd/compile: update PGO profile
7c52c06 
Since last time the default.pgo profile is collected, there has
been a lot of development in the compiler. It's time to refresh
the compiler's PGO profile.

Profile collected by running the cmd/compile/profile.sh script on
the gotip-linux-amd64_c3h88-perf_vs_release gomote.

Benchmark results on Linux/AMD64:

         │  nopgo.txt   │              old.txt               │              new.txt               │
         │    sec/op    │   sec/op     vs base               │   sec/op     vs base               │
Template    120.7m ± 0%   118.1m ± 0%  -2.17% (p=0.000 n=20)   116.5m ± 0%  -3.46% (p=0.000 n=20)
Unicode    104.59m ± 0%   99.05m ± 0%  -5.29% (p=0.000 n=20)   99.30m ± 1%  -5.06% (p=0.000 n=20)
GoTypes     676.8m ± 0%   662.3m ± 0%  -2.15% (p=0.000 n=20)   650.8m ± 0%  -3.85% (p=0.000 n=20)
Compiler    96.29m ± 1%   91.85m ± 1%  -4.61% (p=0.000 n=20)   90.44m ± 1%  -6.08% (p=0.000 n=20)
SSA          5.185 ± 0%    5.038 ± 0%  -2.83% (p=0.000 n=20)    4.945 ± 0%  -4.64% (p=0.000 n=20)
Flate       69.00m ± 0%   67.39m ± 0%  -2.33% (p=0.000 n=20)   66.60m ± 0%  -3.48% (p=0.000 n=20)
GoParser    143.3m ± 0%   140.6m ± 1%  -1.89% (p=0.000 n=20)   138.3m ± 1%  -3.46% (p=0.000 n=20)
Reflect     310.0m ± 1%   305.1m ± 0%  -1.56% (p=0.000 n=20)   298.2m ± 1%  -3.79% (p=0.000 n=20)
Tar         109.5m ± 0%   107.5m ± 1%  -1.79% (p=0.000 n=20)   106.0m ± 0%  -3.18% (p=0.000 n=20)
XML         147.6m ± 1%   144.2m ± 0%  -2.31% (p=0.000 n=20)   142.0m ± 0%  -3.83% (p=0.000 n=20)
geomean     215.0m        209.2m       -2.70%                  206.2m       -4.09%

For #60234.

Change-Id: Iee853146f120ce310c81aabfccf6e5665e6d8b65
Reviewed-on: https://go-review.googlesource.com/c/go/+/587195
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
@dmitshur
Copy link
Contributor

@cherrymui With CL 587195 submitted now, do you expect we might want to do another update before RC 1? When we're confident there's nothing more to do for Go 1.23 milestone, let's move this to the next. Thanks.

At this point it might help to separate out work on making relui do this automatically into a dedicated x/build/cmd/relui issue, since that can happen independently of the Go release cycle.

@cherrymui
Copy link
Member Author

Currently I'm not expecting another update, but I can doublecheck before the RC, in case there are some more changes in the compiler.

@cherrymui cherrymui modified the milestones: Go1.23, Go1.24 Jun 18, 2024
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/634037 mentions this issue: cmd/compile: update default PGO profile

gopherbot pushed a commit that referenced this issue Dec 6, 2024
@cherrymui
cmd/compile: update default PGO profile
3ae00f5 
Since last time the default.pgo profile is collected, there has
been a lot of development in the compiler. It's time to refresh
the compiler's PGO profile.

Profile collected by running the cmd/compile/profile.sh script on
the gotip-linux-arm64_c4ah72-perf_vs_release gomote.

Benchmark results on Linux/AMD64:

         │   nopgo.txt   │              old.txt               │              new.txt               │
         │    sec/op     │   sec/op     vs base               │   sec/op     vs base               │
Template     112.0m ± 1%   109.7m ± 1%  -2.10% (p=0.000 n=20)   110.3m ± 1%  -1.55% (p=0.001 n=20)
Unicode      99.39m ± 0%   94.94m ± 0%  -4.49% (p=0.000 n=20)   94.33m ± 1%  -5.10% (p=0.000 n=20)
GoTypes      544.9m ± 0%   535.0m ± 1%  -1.80% (p=0.000 n=20)   535.1m ± 0%  -1.78% (p=0.000 n=20)
Compiler     96.23m ± 1%   90.86m ± 1%  -5.58% (p=0.000 n=20)   90.84m ± 1%  -5.60% (p=0.000 n=20)
SSA           3.403 ± 1%    3.273 ± 0%  -3.81% (p=0.000 n=20)    3.247 ± 0%  -4.57% (p=0.000 n=20)
Flate        71.55m ± 0%   70.09m ± 1%  -2.04% (p=0.000 n=20)   70.03m ± 1%  -2.13% (p=0.000 n=20)
GoParser     131.5m ± 1%   129.9m ± 1%  -1.19% (p=0.000 n=20)   129.4m ± 0%  -1.56% (p=0.000 n=20)
Reflect      275.5m ± 1%   268.6m ± 1%  -2.50% (p=0.000 n=20)   268.1m ± 1%  -2.70% (p=0.000 n=20)
Tar          131.6m ± 1%   128.4m ± 1%  -2.37% (p=0.000 n=20)   128.8m ± 1%  -2.07% (p=0.000 n=20)
XML          153.7m ± 1%   150.3m ± 1%  -2.26% (p=0.000 n=20)   149.7m ± 1%  -2.66% (p=0.000 n=20)
geomean      200.3m        194.7m       -2.82%                  194.4m       -2.98%

On Linux/ARM64:

         │   nopgo.txt   │              old.txt               │              new.txt               │
         │    sec/op     │   sec/op     vs base               │   sec/op     vs base               │
Template     82.38m ± 1%   80.52m ± 0%  -2.26% (p=0.000 n=20)   80.07m ± 1%  -2.81% (p=0.000 n=20)
Unicode      76.28m ± 0%   72.72m ± 0%  -4.66% (p=0.000 n=20)   72.40m ± 1%  -5.09% (p=0.000 n=20)
GoTypes      420.7m ± 0%   412.5m ± 0%  -1.95% (p=0.000 n=20)   408.1m ± 1%  -3.00% (p=0.000 n=20)
Compiler     69.85m ± 1%   67.40m ± 1%  -3.50% (p=0.000 n=20)   66.73m ± 1%  -4.47% (p=0.000 n=20)
SSA           2.846 ± 0%    2.733 ± 0%  -3.96% (p=0.000 n=20)    2.707 ± 0%  -4.90% (p=0.000 n=20)
Flate        49.14m ± 1%   47.87m ± 0%  -2.58% (p=0.000 n=20)   47.53m ± 1%  -3.26% (p=0.000 n=20)
GoParser     97.44m ± 1%   94.96m ± 1%  -2.55% (p=0.000 n=20)   94.75m ± 1%  -2.76% (p=0.000 n=20)
Reflect      210.5m ± 0%   205.2m ± 0%  -2.50% (p=0.000 n=20)   202.5m ± 0%  -3.78% (p=0.000 n=20)
Tar          97.75m ± 1%   95.72m ± 1%  -2.07% (p=0.000 n=20)   94.68m ± 1%  -3.13% (p=0.000 n=20)
XML          112.6m ± 0%   110.0m ± 0%  -2.33% (p=0.000 n=20)   108.7m ± 1%  -3.49% (p=0.000 n=20)
geomean      150.2m        145.9m       -2.84%                  144.6m       -3.67%

For #60234.

Change-Id: Ie1abc019339252cd9441a1fb1eab33a29c1dc11c
Reviewed-on: https://go-review.googlesource.com/c/go/+/634037
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: David Chase <drchase@google.com>
@mknyszek mknyszek modified the milestones: Go1.24, Go1.25 Dec 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Builders x/build issues (builders, bots, dashboards) NeedsFix The path to resolution is known, but the work has not been done. recurring Issues that should never be closed, but moved to the next milestone once fixed in the current one. release-blocker
Projects
Go Release
Status: Planned
Release Blockers
Status: No status
Milestone
Go1.25
Development

No branches or pull requests
7 participants
@cagedmantis @mknyszek @dmitshur @aclements @gopherbot @cherrymui @heschi