Find good inlining heuristics

[kripken]

We have the ability to inline, and a simple inlining pass using it. That pass inlines only methods called just once, to avoid code size issues. Even that simple rule ends up not having clear benefits in either code size or speed - examples go both ways, so the pass is off by default.

We should investigate this and find good heuristics for inlining. The opportunity here is potentially large, as we can process large compilation units efficiently (and emcc runs us on the whole codebase currently, because of that) so this would mean our inlining is LTO-like in being able to span C compilation units.

[qwertie]

I have no expertise at this, but have a couple of comments. In .NET the JIT will not inline a method of the form...

int Sum(int n1, int n2, int n3, int n4, ...) { return n1 + n2 + n3 + n4 + ... ; }

...if the number of arguments is sufficiently large, so I think they forgot about the "opportunity cost" of not inlining. Inlining eliminates the code that shuffles data into the correct registers and puts excess arguments onto the stack, so the inlining decision should take into account not just the size-cost of inlining but also the size-cost of not inlining.

Here are a couple of thoughts I often have about inlining. I often write functions of this form:

void MaybeDoSomething(int x) {
   if (condition that is rarely true and may or may not involve x) {
      large and expensive procedure
   }
}

I sometimes wish my compiler could inline just the "if" and not the whole procedure.

I also often write internal functions plus public APIs that wrap them, like

public: void InsertItem(int index, T item) {
   if ((size_t)index > _list.size()) 
      throw ....;
   if (this->IsReadOnly())
      throw ...;
   InsertItemCore(index, item);
}
private: void InsertItemCore(int index, T item) {
   _list.Insert(index, foo);
   ... implementation-specific bookkeeping ...
}

My goal is to avoid unnecessary "bureaucratic" checks when I know those checks will succeed. What I really want is a single function with two distinct entry points, but in lieu of that, I want the "Core" function to be inlined in the public function. So sometimes I wish I could insert an inlining hint at a specific call site.

[kripken]

Yeah, it would be interesting to know if cases like those are common. I believe the LLVM inliner has a related heuristic, where it if thinks inlining could get rid of a branch, then it prefers to do it.