Add LTO support (full and thin), with `-flto=thin|full`. #1840

Merged
merged 2 commits into from Nov 9, 2016

Conversation

Projects
None yet
4 participants
@JohanEngelen
Member

JohanEngelen commented Oct 19, 2016

LTO needs linker support: I am only aware of support on OS X and Linux (through the LLVMgold plugin).

I chose the cmdline option flto=thin|full that mimics Clang's option (also Martin was looking for flto in the issue #693 he reported).

Resolves #693 .

For OS X, additional logic could be added later to point to a newer libLTO.dylib (see clang's darwin::Linker::AddLinkArgs). Done: added flto-binary for that purpose. Also, CMake installs LLVM's LTO libs when it can find them; the LTO lib is installed in LDC's /lib/, and that's the first path searched when adding LTO linker flags.

@MartinNowak

This comment has been minimized.

Show comment
Hide comment
@MartinNowak

MartinNowak Oct 21, 2016

Yes, using clang's options seems like the obvious choice.
clang --help | grep -i lto

  -flto=<value>           Set LTO mode to either 'full' or 'thin'
  -flto                   Enable LTO in 'full' mode
  -fno-lto                Disable LTO mode (default)
  -fthinlto-index=<value> Perform ThinLTO importing using provided function summary index

Only have a vague idea what this fthinlto-index option is, also see [ThinLTO] Option to invoke ThinLTO backend passes and importing.

Yes, using clang's options seems like the obvious choice.
clang --help | grep -i lto

  -flto=<value>           Set LTO mode to either 'full' or 'thin'
  -flto                   Enable LTO in 'full' mode
  -fno-lto                Disable LTO mode (default)
  -fthinlto-index=<value> Perform ThinLTO importing using provided function summary index

Only have a vague idea what this fthinlto-index option is, also see [ThinLTO] Option to invoke ThinLTO backend passes and importing.

driver/linker.cpp
+
+ unsigned char opt = optLevel();
+ static char optChars[15] = "-plugin-opt=O0";
+ optChars[13] = '0' + (opt & 0x03); // the plugin O-level ranges from 0 to 3

This comment has been minimized.

@klickverbot

klickverbot Oct 21, 2016

Member

Have you considered just making this min(optLevel(), 3)? -O4 and -O5 are currently documented as being equal to -O3, and it doesn't seem like resetting to 0 resp. 1 would be intuitive for these.

@klickverbot

klickverbot Oct 21, 2016

Member

Have you considered just making this min(optLevel(), 3)? -O4 and -O5 are currently documented as being equal to -O3, and it doesn't seem like resetting to 0 resp. 1 would be intuitive for these.

This comment has been minimized.

@JohanEngelen

JohanEngelen Oct 22, 2016

Member

How stupid, thanks for catching this bug. Trying to optimize something that doesn't need optimizing.

@JohanEngelen

JohanEngelen Oct 22, 2016

Member

How stupid, thanks for catching this bug. Trying to optimize something that doesn't need optimizing.

driver/linker.cpp
+namespace {
+
+std::string getLTOGoldPluginPath() {
+ // TODO: cmdline option for plugin path, or in ldc.conf?

This comment has been minimized.

@klickverbot

klickverbot Oct 21, 2016

Member

Command-line option which can then be set from ldc2.conf? ;)

@klickverbot

klickverbot Oct 21, 2016

Member

Command-line option which can then be set from ldc2.conf? ;)

This comment has been minimized.

@JohanEngelen

JohanEngelen Oct 22, 2016

Member

Note to self: the plugin path is also useful for OS X.

@JohanEngelen

JohanEngelen Oct 22, 2016

Member

Note to self: the plugin path is also useful for OS X.

@JohanEngelen

This comment has been minimized.

Show comment
Hide comment
Member

JohanEngelen commented Oct 23, 2016

@MartinNowak See http://blog.llvm.org/2016/06/thinlto-scalable-and-incremental-lto.html for -fthinlto-index . I'm leaving that out for now.

@JohanEngelen

This comment has been minimized.

Show comment
Hide comment
@JohanEngelen

JohanEngelen Oct 24, 2016

Member

Bug : When there are module ctors/dtors in different modules, not all of them are run :(

Edit: I think I found the cause. There is a latent bug in our llvm.used emission. ThinLTO is very aggressively optimizing and thus things break. Working on a fix. (edit: #1855)

Edit2: nice! With that fix, I managed to build LDC with LTO, mixing C++ and D LTO! In other words: cross-language cross-module inlining.

Member

JohanEngelen commented Oct 24, 2016

Bug : When there are module ctors/dtors in different modules, not all of them are run :(

Edit: I think I found the cause. There is a latent bug in our llvm.used emission. ThinLTO is very aggressively optimizing and thus things break. Working on a fix. (edit: #1855)

Edit2: nice! With that fix, I managed to build LDC with LTO, mixing C++ and D LTO! In other words: cross-language cross-module inlining.

@kinke

This comment has been minimized.

Show comment
Hide comment
@kinke

kinke Oct 24, 2016

Member

mixing C++ and D LTO

Pretty cool. :)

Member

kinke commented Oct 24, 2016

mixing C++ and D LTO

Pretty cool. :)

@JohanEngelen

This comment has been minimized.

Show comment
Hide comment
@JohanEngelen

JohanEngelen Oct 27, 2016

Member

LTO testing is now enabled on CircleCI

Member

JohanEngelen commented Oct 27, 2016

LTO testing is now enabled on CircleCI

@JohanEngelen

This comment has been minimized.

Show comment
Hide comment
@JohanEngelen

JohanEngelen Nov 1, 2016

Member

Any concerns before I merge this?

Member

JohanEngelen commented Nov 1, 2016

Any concerns before I merge this?

JohanEngelen added some commits Oct 16, 2016

Add LTO support (full and thin), with `-flto=thin|full`.
LTO needs linker support: I am only aware of support on OS X and Linux (through the LLVMgold plugin).

Resolves #693
@JohanEngelen

This comment has been minimized.

Show comment
Hide comment
@JohanEngelen

JohanEngelen Nov 9, 2016

Member

Merging when green.

Member

JohanEngelen commented Nov 9, 2016

Merging when green.

@JohanEngelen JohanEngelen merged commit 3722caa into ldc-developers:master Nov 9, 2016

3 checks passed

ci/circleci Your tests passed on CircleCI!
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@JohanEngelen JohanEngelen deleted the JohanEngelen:LTO branch Nov 10, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment