Profile-Guided Optimization (PGO) questions

[zamazan4ik]

Hi!

I am investigating the Profile-Guided Optimization (PGO) state across the industry - all my current results are available at https://github.com/zamazan4ik/awesome-pgo . After researching a little bit LDC's PGO information I have several questions.

1. What is the most up-to-date place for LDC documentation? Am I right that this is the most updated PGO-related page for LDC?
2. Does LDC support Sampling PGO (also known as AutoFDO)? If Sampling PGO is not supported - do you plan to support it in the future? For us sampling PGO can be important since it's much easier to use for gathering the PGO profiles directly from the production environment without hurting the production performance a lot.
3. Do you support other PGO modes like CSIR PGO in LDC? If no, do you plan to support them in the future?
4. What performance improvements did you get with enabling LTO + PGO on LDC? Could you please share the number? With this information it's much easier to consider rebuilding LDC (due to strict security requirements) locally with PGO since we can estimate benefits from PGO for LDC based on the actual benchmarks from the LDC developers.
5. Is there any documentation on how to build LDC with LTO+PGO? I am looking for smth like it's done in the ClickHouse documentation.
6. Am I right that the officially released LDC binaries are already LTO + PGO optimized? According to the script they are but wanted just to recheck this information :)

Thanks a lot for the help!

[JohanEngelen]

1. We don't really have good documentation here. The wiki page you refer to does not contain all info. Best to look at clang documentation, and check if LDC has the commandline option needed (--help).
2. Sampling PGO not yet. It will be probably be very easy to add though. Maybe I'll get around to implementing it this year, or someone else ;)
3. We support front-end PGO, and IR PGO. But not CSIR PGO (probably will also be easy to implement, because LLVM provides the functionality). Please file an issue so we don't forget
4. When LDC is built with LTO+PGO it gives 10-15% faster compilation time on complex codebase at Weka.io (the PGO profile is made by compiling Phobos/druntime, not by compiling the weka code). I did not do extensive measurements, and by now I forgot the difference between only PGO, only LTO, and LTO+PGO.
5. Not that I know of. See 6.
6. Yes.

[zamazan4ik]

Just adding here reference to the relevant discussion on the D forum: https://forum.dlang.org/post/aazvttfcplcdghxlvcqi@forum.dlang.org

[cyrusmsk]

(sometimes forum could be down - so I'm also adding materials here)

Some internal details about PGO is in this article: https://archive.fosdem.org/2017/schedule/event/ldc_d_optimization/attachments/slides/1819/export/events/attachments/ldc_d_optimization/slides/1819/FOSDEM_2017.pdf

and blog posts from LDC dev:
http://johanengelen.github.io/ldc/2016/11/10/Link-Time-Optimization-LDC.html
http://johanengelen.github.io/ldc/2016/04/13/PGO-in-LDC-virtual-calls.html

For PGO + LTO example of usage and the performance improvements, I think one of the most helpful source is this repo:

https://github.com/eBay/tsv-utils/blob/master/docs/BuildingWithLTO.md
https://github.com/eBay/tsv-utils/blob/master/docs/lto-pgo-study.md