D variadic codegen bad for x86 32-bit and LLVM 3.6

[smolt]

A D variadic function call produced by LLVM 3.6 leads to bad stack adjustments post function call and causes the program to crash. Specific to LLVM 3.6 because I can rebuild LDC 0.15.2 with LLVM 3.5.1 and all is good.

Can be observed on OS X with release 0.15.2 beta2 download - built with LLVM 3.6.1.
Demo:

void vfun(int x, ...) {}

void main()
{
    vfun(1,2,3,4);
}

ldc2 -m32 -O vararg.d
$ ./vararg
Segmentation fault: 11

I traced with debugger and I can see that stack is wrong after all new adjusts have been made post function call.

The assembly difference between 3.6.1 and 3.5.1 is minimal. Here is 3.6.1 asm with comments to show what 3.5.1 does differently:

__D6vararg4vfunFiYv:
    pushl   %eax
    leal    20(%esp), %ex
    movl    %eax, (%esp)
    popl    %eax
    retl    $12   // LLVM 3.5.1 does simple retl without stack adjust

    .globl  __Dmain
    .align  4, 0x90
__Dmain:
    subl    $28, %esp
    movl    $4, 20(%esp)
    movl    $3, 16(%esp)
    movl    $2, 12(%esp)
    calll   __D6vararg4vfunFiYv
    subl    $24, %esp     // LLVM 3.5.1 does not produce any adjust here
    xorl    %eax, %eax
    addl    $28, %esp
    retl

Ignore the fact that optimization didn't remove the entire call. Same bad stack adjustment with and without the optimizer.

I also believe this affects other Posix and think this is the reason for the unittest failures on Travis with LLVM 3.6 and 32-bit.

https://travis-ci.org/ldc-developers/ldc/jobs/69629942

These are the unittests that fail when I build LDC against LLVM 3.6.1 on OS X:

 19 - core.thread (SEGFAULT)
204 - std.datetime (SEGFAULT)
218 - std.outbuffer (SEGFAULT)
231 - std.stream (SEGFAULT)
236 - std.typecons (SEGFAULT)
499 - std.outbuffer-debug (SEGFAULT)
512 - std.stream-debug (SEGFAULT)
517 - std.typecons-debug (SEGFAULT)

[smolt]

LLVM 3.7.0svn (master today), issue remains.

BTW - I think only unittests failures above due to this are std.outbuffer, std.stream, and std.typecons.

P.S. Studying it some more, it appears the wrong stack adjustment matches the number of varargs bytes on the stack, as if they had been pushed. But the caller has preallocated stack so no pushing was involved.

[dnadlinger]

core.thread is unrelated, it also happens on x86_64.

[smolt]

Yeah - core.thread is your #666.

[dnadlinger]

Fixed in master.