DI Generation Improvements. #945

LightBender · 2012-05-13T07:19:41Z

This pull request improves the follow DI generation rules:

All functions stripped of their implementations except for template functions and auto return functions.
Improved indenting.

Requires: dlang/druntime#218
Supersedes: #538

Fix issue: http://d.puremagic.com/issues/show_bug.cgi?id=1427
Fix issue: http://d.puremagic.com/issues/show_bug.cgi?id=5461

andralex · 2012-09-25T16:07:02Z

@LightBender time to rebase and see what happens with the tester failures!

WalterBright · 2012-11-11T03:22:57Z

By stripping all the function bodies out, then functions in .di files lose the ability to be inlined. I disagree with this change.

I am also not seeing much point in pretty-printing .di files. If someone wants to build a D source code formatting filter, that would be the place for it. Then, the .di file could be piped through that. Otherwise, we'll be building a pretty-printer twice.

LightBender · 2012-11-11T03:28:33Z

You realize that without this change D is pretty much useless for commercial library development as ALL function implementations are included in the DI file. It would be like writing your entire library in a header file. It doesn't matter if you're an OSS guy, but legal teams at commercial shops everywhere are going to scream bloody murder...

Don't expect any commercial libraries to be built for D without this.

WalterBright · 2012-11-11T03:57:08Z

On 11/10/2012 7:28 PM, LightBender wrote:

You realize that without this change D is pretty much useless for commercial
library development as ALL function implementations are included in the DI file.

Just the ones that look like they are inlineable. Functions that have loops in
them are all not considered inlineable.

It would be like writing your entire library in a header file. It doesn't
matter if you're an OSS guy, but legal teams at commercial shops everywhere
are going to scream bloody murder...

Don't expect any commercial libraries to be built for D without this.

These days, an increasing amount of the code is winding up as templates anyway,
which simply must be included. But, that said, I understand the desire to hide
the function bodies. To that end, there are still possibilities:

delete them manually. Ok, it's tedious, but it's not more work than it is for
C++.
use PIMPL (pointer to implementation) style, where all those .di functions do
is transfer control elsewhere to hidden functions.

And, if someone is building a commercial library and considers themselves
blocked by this, please contact me.

andralex · 2012-11-12T20:45:24Z

I discussed this with Walter. There are a few reasonable views on this:

Separate compilation is what it is, so do as Adam suggests.
For the most part short, inlineable functions are unlikely to change often and efficiency is what it is, so keep inlineable function bodies in the .di file.
Give people a possibility to direct inlining. A possibility raised by Walter is to force inlined functions to be templates by simply adding a "()" prior to the parameter list. That would put them in the .di file without otherwise affecting things (there are, however, some oddities such as overload resolution that I suspect will create puzzlement down the road). One other possibility that I just figured is to put in the .di by definition all functions that return "auto". I like this a whole lot better. A third possibility is to add a new attribute such as @alwaysinline or pragma(inline) to control inlining. Walter and I prefer working with the features we have instead of adding new ones. My vote goes to use of auto returns to control inlining.

Regarding the exchange: (a) please let's not overstate the case; (b) using C++ as a baseline is inadequate; (c) "if someone is building a commercial library and considers themselves blocked by this, please contact me" is rather provincial and non-scalable - it would behoove us to never talk or think that way.

alexrp · 2012-11-12T20:50:13Z

@andralex @alwaysinline has been requested countless times by people on the newsgroup because they actually know what they're doing, but it keeps being either ignored or dismissed as unnecessary, which I think is absolute nonsense for a systems language. Most of these people now resort to compiler-specific attributes that GDC and LDC provide. This is a pretty silly situation when all compilers can relatively easily support it.

So, you know, there's that. Adding that attribute would kill two birds with one stone.

andralex · 2012-11-12T21:25:05Z

Again, I'm favoring using the current language over adding new features.

WalterBright · 2012-11-12T23:41:56Z

How about this:

If the switch -inline is used for the .di generation, then the bodies are included. Otherwise, not.

andralex · 2012-11-12T23:59:18Z

Like

dnadlinger · 2012-11-13T00:16:49Z

@andralex: Dislike. ;)

In my opinion, there is no good reason from a user's perspective why -inline should influence .di generation, as it is purely a code generation flag. Whether I use -inline to build my library is not related to whether I want function bodies to be kept when generating the import modules. As a litmus test, think about how you would change the description of the -inline flag if this behavior was implemented: »do function inlining; also keep function bodies on 'header' generation«?

I think this makes it clear that the switches have nothing to do with each other. There is nothing wrong in adding an additional command line flag for the .di generator (-Hkeep-function-bodies or whatever cryptic name instead), if, well, a toggle-able feature is added to the .di generator. If you are worried about the perceived complexity of the command line interface, add a separate »header generation« subsection to the help and group all the switches there. DMD certainly doesn't become less complex from a user's perspective when command line switches are arbitrarily overloaded with different meanings.

andralex · 2012-11-13T00:22:37Z

I think it's very fit to have -inline mean keep inlined function bodies when generating .di files. Yes, it is a different meaning, but it's right on the spot. I don't see why people should remember a different flag when it comes about the same concept, just applied to .di generation.

dnadlinger · 2012-11-13T00:25:47Z

How is »do function inlining« related to »keep function bodies«? How are two meanings for the same flag easier to remember than two well-named ones? If you knew nothing about D, would dmd -H -inline <…> or dmd -H -keep-function-bodies <…> rather lead you to a correct guess of the intent of the invocation?

WalterBright · 2012-11-13T00:36:06Z

The only reason to keep the function bodies is to enable inlining of them.
.di generation is done separately from code generation, so it is reasonable that the -inline flag pertain to it
-di -inline means "I want to generate .di files and support inlining of functions in it". It sounds straightforward.

dnadlinger · 2012-11-13T00:37:19Z

@andralex: Regarding the previous question, I think it is a proven fact that being able to override the compiler's inline heuristics is a useful feature in certain, rare cases. One of these is certainly real-time graphics/game engines, where you often can't afford a function call to be inserted for math code, even in debug mode. For example, Manu/@TurkeyMan has asked several times for something like this on the newsgroup.

Note that this has nothing to do with the C++ inline attribute which is indeed more or less an exercise in futility. pragma(alwaysInline); (which is my preferred syntax), just like the vendor-specific forceinline/always_inline attributes in C compilers, is a sharp tool for solving specific performance problems, which would be hard to attack otherwise. It is not intended to be used frequently, and you can easily shoot yourself in the foot performance-wise if you apply it without proper benchmarking. It's a compiler pragma, not a language feature, for a reason.

However, contrary to @alexrp, I don't think this should have anything to do with the problem discussed here, other than the fact that the .di generator could keep function bodies which are marked pragma(alwaysInline). The simple reason for this is that for the vast majority of cases where inlining entails a performance bit, alwaysInline will not be used, by design.

andralex · 2012-11-13T00:42:31Z

I'm totally with Walter on 1-3, he essentially spelled it for me. @klickverbot you're making a good point about forcing inline but let's leave that to a future diff. @LightBender could you please do the honors of implementing the decision depending on -inline? Thanks all!

dnadlinger · 2012-11-13T00:45:14Z

@WalterBright: What do you mean by ».di generation is done separately from code generation«? If you mean on a compiler level, then obviously yes, I think I'm somewhat familiar with that. If you mean that it is done separately as in not in the same compiler invocation in which code is generated, then that's not always the case. Apparently quite a few people use -H along with compilation (I discovered that when investigating an LDC bug), which actually makes sense when building/installing open source packages (and for whatever reason you assume that it is beneficial to install .di files, which is questionable).

The fundamental problem I have with the idea is that -inline seems to mean »do something now«, whereas keeping the function bodies around enables something else to be done later. Why doesn't -di -inline mean »I want to generate .di files and have inlining of functions performed«?

I still think there is no good reason to conflate the two concepts. You probably wouldn't expect -finline-functions to keep around function bodies either if GCC supported an analogous feature. I for sure wouldn't. If you think this makes the compiler simpler to understand, then apparently our definitions of »simple« differ…

WalterBright · 2012-11-13T01:07:58Z

@klickverbot, you're right, Manu's feature request is distinct from this one. "alwaysinline" is not the negation of "neverinline".

alexrp · 2012-11-13T06:46:39Z

The only reason to keep the function bodies is to enable inlining of them.

... and CTFE.

TurkeyMan · 2012-11-13T15:46:24Z

... and CTFE.

And this is rather a big deal.

dnadlinger · 2012-11-13T17:47:14Z

Hm, regarding CTFE, wouldn't .di generation simply not be used at all in that case?

Also, please note that while I still don't see why merging the additional flag into -inline would be beneficial, I don't think the issue is important enough to warrant lengthy discussion. So, if others agree, please go ahead and implement it like that. Originally, I only intended to make sure that the (non-)relation between alwaysInline and the discussed topic is clarified.

Issue 9150 - Mismatching static array length should be detected in foreach

…r function bodies.

…put.

…ementations with -inline specified. Added unittests for DMD for the -inline switch.

LightBender · 2013-01-09T07:12:52Z

I have added the -inline switch support, new unittests for DMD covering the new -inline switch, and cleaned up some dead code that was used in earlier iterations of this pull request.

This is now ready to merge.

I want to give a shout-out to @AndrejMitrovic for helping me out with the Unittests.

WalterBright · 2013-01-09T09:27:11Z

src/root/root.c

@@ -1545,10 +1545,18 @@ void OutBuffer::prependstring(const char *string)
 void OutBuffer::writenl()
 {
 #if _WIN32
+#if M_UNICODE


There's no reason to support M_UNICODE. None of the rest of the code does.

Ok, i'll remove it. I think that was a hold-over from when I copied kenji's old pretty printing pull because I didn't write that.

WalterBright · 2013-01-09T10:35:43Z

src/func.c

-    if (fbody &&
-        (!hgs->hdrgen || hgs->tpltMember || canInline(1,1,1))
-       )
+    if (fbody && (!hgs->hdrgen || global.params.useInline || hgs->autoMember || hgs->tpltMember))
    {   buf->writenl();



In here:

save values of autoMember and tpltMember

set autoMember and tpltMember to 0

execute code in { }

restore autoMember and tpltMember to their saved values

LightBender · 2013-01-10T03:45:11Z

src/func.c

+        int saveauto = hgs->autoMember;
+        hgs->tpltMember = 0;
+        hgs->autoMember = 0;
+


@WalterBright Is this what you had in mind?

yes, that works

LightBender · 2013-01-15T07:45:48Z

Due to some irretrievable merge conflicts in the last rebase I have had to close this pull and start again. See pull: #1487 for the new pull request.

As discussed in the original PR that added the feature (dlang#945), DMD currently triggers two entirely unrelated features with the -inline switch. Since LDC does not really have an equivalent to that switch in the first place, but users still want to control the emission of function bodies, we need to be control it separately. For DMD, global.params.hdrStripPlainFunctions is always set to !global.params.useInline in the driver to match the current behavior.

As discussed in the original PR that added the feature (dlang#945), DMD currently triggers two entirely unrelated features with the -inline switch. Since LDC does not really have an equivalent to that switch in the first place, but users still want to control the emission of function bodies, we need to be able to control it separately. For DMD, global.params.hdrStripPlainFunctions is always set to !global.params.useInline in the driver to match the current behavior.

Adam Wilson added 2 commits June 26, 2012 20:48

Initial DI generation improvements.

8734374

Fixed merge conflict in HEAD.

281b47f

Don Clugston and others added 8 commits December 13, 2012 08:43

Merge pull request #1373 from 9rnsr/fix9150

71460c1

Issue 9150 - Mismatching static array length should be detected in foreach

Initial DI generation improvements.

98c696b

Fixed merge conflict in HEAD.

a640f8f

Rebased and fixed merge conflicts.

bbb496b

Fixed merge conflicts with origin.

9e95fb4

Will now use the -inline switch to determine if DI files include thei…

eb860f5

…r function bodies.

Updated the extra-files header.di and xheader.di to match the new out…

d735ad8

…put.

Removed the generated by comment.

b82bdaf

Adam Wilson added 4 commits January 8, 2013 19:52

Removed the dead isFuncDeclaration in attrib

12850f7

Fixed a bug that would cause some functions to not receive their impl…

a62f384

…ementations with -inline specified. Added unittests for DMD for the -inline switch.

Do not need permute args for inlineheader.d test.

a19b37f

Set the inlineheader unittest postscripts modes to +x.

17f88c2

WalterBright reviewed Jan 9, 2013
View reviewed changes

Removed the extraneous unicode support from writenl()

62dead4

WalterBright reviewed Jan 9, 2013
View reviewed changes

Nested functions will now be correctly handled (See Walters comments).

7a3a17e

LightBender reviewed Jan 10, 2013
View reviewed changes

Updated header.di test to account for the nested function problem.

48c7a2f

LightBender mentioned this pull request Jan 15, 2013

DI Generation Improvements. #1487

Merged

LightBender closed this Jan 15, 2013

LightBender deleted the digen branch January 15, 2013 07:49

dnadlinger mentioned this pull request May 6, 2016

Split global.params.hdrStripPlainFunctions from useInline #5733

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DI Generation Improvements. #945

DI Generation Improvements. #945

LightBender commented May 13, 2012

andralex commented Sep 25, 2012

WalterBright commented Nov 11, 2012

LightBender commented Nov 11, 2012

WalterBright commented Nov 11, 2012

andralex commented Nov 12, 2012

alexrp commented Nov 12, 2012

andralex commented Nov 12, 2012

WalterBright commented Nov 12, 2012

andralex commented Nov 12, 2012

dnadlinger commented Nov 13, 2012

andralex commented Nov 13, 2012

dnadlinger commented Nov 13, 2012

WalterBright commented Nov 13, 2012

dnadlinger commented Nov 13, 2012

andralex commented Nov 13, 2012

dnadlinger commented Nov 13, 2012

WalterBright commented Nov 13, 2012

alexrp commented Nov 13, 2012

TurkeyMan commented Nov 13, 2012

dnadlinger commented Nov 13, 2012

LightBender commented Jan 9, 2013

WalterBright Jan 9, 2013

LightBender Jan 9, 2013

WalterBright Jan 9, 2013

LightBender Jan 10, 2013

WalterBright Jan 10, 2013

LightBender commented Jan 15, 2013

DI Generation Improvements. #945

DI Generation Improvements. #945

Conversation

LightBender commented May 13, 2012

andralex commented Sep 25, 2012

WalterBright commented Nov 11, 2012

LightBender commented Nov 11, 2012

WalterBright commented Nov 11, 2012

andralex commented Nov 12, 2012

alexrp commented Nov 12, 2012

andralex commented Nov 12, 2012

WalterBright commented Nov 12, 2012

andralex commented Nov 12, 2012

dnadlinger commented Nov 13, 2012

andralex commented Nov 13, 2012

dnadlinger commented Nov 13, 2012

WalterBright commented Nov 13, 2012

dnadlinger commented Nov 13, 2012

andralex commented Nov 13, 2012

dnadlinger commented Nov 13, 2012

WalterBright commented Nov 13, 2012

alexrp commented Nov 13, 2012

TurkeyMan commented Nov 13, 2012

dnadlinger commented Nov 13, 2012

LightBender commented Jan 9, 2013

WalterBright Jan 9, 2013

Choose a reason for hiding this comment

LightBender Jan 9, 2013

Choose a reason for hiding this comment

WalterBright Jan 9, 2013

Choose a reason for hiding this comment

LightBender Jan 10, 2013

Choose a reason for hiding this comment

WalterBright Jan 10, 2013

Choose a reason for hiding this comment

LightBender commented Jan 15, 2013