Evaluate using Profile-Guided Optimization (PGO) and Post-Link Optimization (PLO)

[zamazan4ik]

Hi!

Recently I checked Profile-Guided Optimization (PGO) improvements on multiple projects. The results are available here. According to the tests, PGO can help to achieve better performance in many cases for many applications including compilers like Clang, GCC, Rustc, LDC, Go, etc. I think trying to optimize Shady/VCC with PGO can be a good idea.

I can suggest the following action points:

• Perform PGO benchmarks on Shady. If it shows improvements - add a note to the documentation about possible improvements in Shady's performance with PGO.
• Providing an easier way (e.g. a build option) to build scripts with PGO can be helpful for the end-users and maintainers since they will be able to optimize Shady according to their workloads.
• Optimize pre-built Shady binaries (if any).

Testing Post-Link Optimization (PLO) techniques (like LLVM BOLT) would be interesting (Clang and Rustc already use BOLT as an addition to PGO) but I recommend starting from the usual PGO.

Here are some examples of how PGO optimization is integrated into other projects:

• Rustc: a CI script for the multi-stage build
• GCC:
  • Official docs, section "Building with profile feedback"
  • A part
• Clang: Docs
• Python:
  • CPython: README
  • Pyston: README
• Go: Bash script
• V8: Bazel flag
• ChakraCore: Scripts
• Chromium: Script
• Firefox: Docs
• PHP - Makefile command and old Centminmod scripts
• MySQL: CMake script
• YugabyteDB: GitHub commit
• FoundationDB: Script
• Zstd: Makefile
• Foot: Scripts
• Windows Terminal: GitHub PR
• Pydantic-core: GitHub PR
• file.d: GitHub PR
• OceanBase: CMake flag

Here are some examples of how PGO-related documentation could look in the project:

• ClickHouse: https://clickhouse.com/docs/en/operations/optimizing-performance/profile-guided-optimization
• Databend: https://databend.rs/doc/contributing/pgo
• Vector: https://vector.dev/docs/administration/tuning/pgo/
• Nebula: https://docs.nebula-graph.io/3.5.0/8.service-tuning/enable_autofdo_for_nebulagraph/
• GCC: Official docs, section "Building with profile feedback" (even AutoFDO build is supported)
• Clang:
  • https://llvm.org/docs/HowToBuildWithPGO.html
  • https://llvm.org/docs/AdvancedBuilds.html
• tsv-utils: https://github.com/eBay/tsv-utils/blob/master/docs/BuildingWithLTO.md

Below are listed some LLVM BOLT results:

• Rustc:
  • Rustc itself (GitHub PR)
  • LLVM in Rustc (Reddit)
• CPython: GitHub PR
• YDB: GitHub comment
• Clang:
  • Slides
  • Results on building Clang
  • Linaro results
  • on AMD 7950X3D
  • GitHub comment from LLVM
• LDC: GitHub comment
• HHVM, Proxygen and others: Facebook paper
• NodeJS: Blog
• Chromium: Blog
• MySQL, MongoDB, memcached, Verilator: Paper

I would be happy to answer your questions about PGO and PLO.

I understand that the project is in the early development stages so features could have a bigger priority than performance. Anyway, in this case, you can treat the issue just like an idea for future improvements.

[Hugobros3]

This is basically spam, I'm going to close this.

If you find some interesting performance pitfalls using those tools and want to raise a specific issue, or better yet want to contribute, feel free to open a separate issue.

[kloczek]

If this ticket suggests that vcc source code can/could be somehow reshaped using some data generated PGO IMO it would be good to try to at least try to make few experiments.
Q: Alexander is that was your intention on opening this ticket? 🤔

[zamazan4ik]

If this ticket suggests that vcc source code can/could be somehow reshaped using some data generated PGO IMO it would be good to try to at least try to make few experiments.

Even if PGO can be used to get some insights about some possible code tweaks (the first thing that appears in my mind - getting insights about code inlining) I do not recommend doing it. PGO is created with the idea do not spend/waste time on such optimizatons and allowing the compiler to perform them in a better way (based on the runtime profile).

Alexander is that was your intention on opening this ticket?

Nope. My suggestion with this ticket is about building VCC with PGO so VCC will compile programs faster. I have shown you that many major compilers (including Clang) are already PGO (and PLO) optimized - they getting near +20% compilation speed. And, honestly, I do not understand why such a suggestion to the project is treated as spam. Yes, I created similar (but not equal) in many other projects. But it's only because I am interested in improving performance in many solutions, and PGO helps with it in many cases (as it's proven with my and not only mine tests).

However, I want to highlight my point. If VCC/Shady authors/maintainers think that PGO is not worth it or you just do not care about the VCC/Shady performance - it's completely fine! Every project is completely free to choose the priorities for them. I just put an idea, nothing more.

P.S. IMHO, the better place for creating such ideas is the Discussion functionality on GitHub, in the "Ideas" category. However, since Discussions are disabled right now, I created the issue instead.

[Hugobros3]

Opening hundreds of issues with the same copy-pasted point, unprompted, no matter how "good" your intentions are, is definitely spam.Trying to force the issue when it's clear my time is limited, and inviting drive-by comments gets this convo locked.

I've clarified already I'm open to concrete contributions that either point out or fix actual problems.