fix ldc optimization settings

[MartinNowak]

Update ldc optimization flags to match ldmd behavior.

Just ran into a giant performance issue (50%) while benchmarking Mir, b/c I was using --compiler=ldc2 instead of --compiler=ldmd2.

[MartinNowak]

Wait, this doesn't seem to be everything.
@John-Colvin says -singleobj is used by ldmd, but seems difficult to translate to dub.
https://github.com/ldc-developers/ldc/blob/2dec29b82507115ec4380cecbad118703646a897/driver/ldmd.cpp#L796

[coveralls]

Coverage remained the same at 61.354% when pulling 81726df on ldc-optimization into f7b4db4 on master.

[MartinNowak]

Let's leave -singleobj to be added by users or ldc people.

[MartinNowak]

Summoning @klickverbot and @JohanEngelen for their opinion.

[JohanEngelen]

-singleobj is definitely extremely important for optimization. I am working on cross-module inlining (which is what -singleobj will give you for all the modules passed in one LDC invocation), but we disabled it per default for now because there are subtle issues that result in link errors.
I don't know enough about Dub to contribute much here though. Just a strong recommendation to add it to the default flags.

[MartinNowak]

-singleobj is definitely extremely important for optimization. I am working on cross-module inlining (which is what -singleobj will give you for all the modules passed in one LDC invocation), but we disabled it per default for now because there are subtle issues that result in link errors.

Dub build static libraries by default so that wouldn't work. I was very suprised to learn that ldmd2 and ldc2 behave differently w/ regards to -singleobj.

I am working on cross-module inlining

Yes, Kenji made that work for dmd.

Also what's the state of LTO support, and what are good places to learn about such ldc internals (newsgroup, github)?

[JohanEngelen]

what's the state of LTO support

You can compile to LLVM bitcode (-output-bc), and then input that into LDC. LDC accepts .bc files on the commandline and inputs them all into one module.
LLVM's ThinLTO support is a WIP. You'll certainly hear about it on the forums. It mostly depends on the capabilities of the user's linker though, LDC support is easily added (and almost done). The just-released XCode 8 can do ThinLTO, which means I've started looking into it again.

[s-ludwig]

We could probably put something in BuildGenerator to issue -singleobj for executable targets. Then the user would have the option to use -combined to get a fully optimized build. The question is just where to best put the means for opting out of -singleobj.

Should we pursue this in this context or just merge the basic optimization flags for now?

[dnadlinger]

The question is just where to best put the means for opting out of -singleobj.

There is not really any reason to opt out of it for. In fact, -singleobj has been the default for ldmd2 for ages and will finally become the default for ldc2 with the next release as well.

I was very suprised to learn that ldmd2 and ldc2 behave differently w/ regards to -singleobj.

It's a historical accident. -singleobj was introduced only later in LDC's life, and we didn't want to break backwards-compatibility without being sure about the tradeoffs.

Yes, Kenji made that work for dmd.

The situation is entirely different for us, though, as we are not using DMD's front-end inliner. The latter just doesn't hit a lot of the AST bugs where symbol names depend on the codegen order because it can't handle any remotely complex code anyway. We emit the functions in question as normal to LLVM IR with a special linkage to designate them as for inlining only, and then leave the rest to LLVM's inliner. This way, we are very general, but hit a lot of the AST inconsistencies.

[MartinNowak]

Should we pursue this in this context or just merge the basic optimization flags for now?

Let's optimize the other ones for now. I'd also argue that -singleobj is a problematic workaround, and we should be able to achieve the same results with LTO.
People could use ldc-flags -singleobj if ldmd wouldn't incorrectly match the ldc prefix (a duplicate -singleobj causes an error).

[s-ludwig]

There is not really any reason to opt out of it for. In fact, -singleobj has been the default for ldmd2 for ages and will finally become the default for ldc2 with the next release as well.

Okay, I was basing this on Johan's remark about it possibly causing linker errors. Can those be considered solved by now, also for static libraries? In that case I think we can just add it as a default flag (except for single-file compilation mode perhaps) and be done with it.

[JohanEngelen]

@s-ludwig The linker errors happen for cross-module inlining, but it is default disabled, don't worry about it. It is not related to -singleobj.

[s-ludwig]

Ah okay, I read that sentence too fast. In that case I'd suggest to simply replace the old workaround at

dub/source/dub/compilers/ldc.d

Line 83 in 3efb665

settings.addDFlags("-oq", "-od=.dub/obj");

with -singleobj.