Bug 6376: Segfault on Mac OS X 10.7 when throwing

[kennytm]

Fixes (?) bug 6376: Throwing exception or assertion failure causes segfault or memory error due to ASLR on Mac OS X 10.7.

The patch reduces the size of the DefaultTraceInfo class from > 4 KiB to 2 KiB (can be less), which mysteriously allow the program not segfault anymore (at least for the druntime and Phobos unit tests). I don't know why this happens, but _probably_ because if it is longer, the class instance starts to override the original Throwable's classinfo.

The actual fix is commit 15706b2, the other one (8bce1bd) is just some changes to make the files compilable without -d and fix some signatures, which is not necessary.

[braddr]

Before I'm comfortable with this change, we'll need to understand why it
helps. I'm happy that there's at least something that makes it go away,
which should help narrow down figuring out the why part, but until the why
is understood, it's not certain that the problem is actually fixed.

On Mon, 25 Jul 2011, kennytm wrote:

Date: Mon, 25 Jul 2011 11:46:40 -0700
From: kennytm
<reply@reply.github.com
>
To: braddr@puremagic.com
Subject: [druntime] Bug 6376: Segfault on Mac OS X 10.7 when throwing (#42)

Fixes (?) bug 6376: Throwing exception or assertion failure causes segfault or memory error due to ASLR on Mac OS X 10.7.

The patch reduces the size of fixbuf member of the DefaultTraceInfo class from 4096 to 2028 (can be less), which mysteriously allow the program not segfault anymore (at least for the druntime and Phobos unit tests). I don't know why this happens, but _probably_ because if it is longer, the class instance starts to override the original Throwable's classinfo.

The actual fix is commit 5a706ad, the other one (4607c36) is just some changes to make the files compilable without -d and fix some signatures, which is not necessary.

Reply to this email directly or view it on GitHub:
#42

[dnadlinger]

Maybe I'm missing something obvious, but do you have any idea how a bug fixed this could be related to ASLR? I'll test the workaround with my code and report back.

[kennytm]

@braddr, @klickverbot: I agree, but I don't know the reason yet. For now I can only say, experimentally, when the TraceInfo's class instance size exceeds 2048, the __vptr of the Throwable object will be corrupted, which make its classinfo become invalid and all exceptions isn't catchable and ignored, leading all sorts of weird outcomes.

---

Just for completeness, my test case:

import core.stdc.stdio;
import core.stdc.stdlib;

version = ChangeClassInfo;

extern (C) void rt_setTraceHandler(Throwable.TraceInfo function(void* ptr));

class DefaultTraceInfo : Throwable.TraceInfo {
    override int opApply( scope int delegate(ref char[]) dg ) {
        return 0;
    }
    override int opApply( scope int delegate(ref size_t, ref char[]) dg ) {
        return 0;
    }
    override string toString() {
        return "lol";
    }

    version(ChangeClassInfo)
        ubyte[2037] fixbuf;
    else
        ubyte[2036] fixbuf;
}

Throwable.TraceInfo defaultTraceHandler2( void* ptr = null ) {
    return new DefaultTraceInfo;
}

void main() {
    auto f = &defaultTraceHandler2;
    rt_setTraceHandler(f);

    auto g = new Exception("");
    auto gptr = cast(void***) g;

    printf("sizeof(DefaultTraceInfo) == %zu\n", __traits(classInstanceSize, DefaultTraceInfo));

    printf("%p; deref=%p, deref2=%p, classinfo=%p\n", gptr, *gptr, **gptr, g.classinfo);
    try {
        throw g;
    } finally {
        printf("%p; deref=%p, deref2=%p, classinfo=%p\n", gptr, *gptr, **gptr, g.classinfo);
        exit(0);
    }
}

Example of normal (2048) result:

sizeof(DefaultTraceInfo) == 2048
0x201f80; deref=0x5825c, deref2=0x5820c, classinfo=0x5820c
0x201f80; deref=0x5825c, deref2=0x5820c, classinfo=0x5820c

Example of corrupted (2052) result:

sizeof(DefaultTraceInfo) == 2052
0x301f80; deref=0xc126c, deref2=0xc121c, classinfo=0xc121c
0x301f80; deref=0x301f40, deref2=0x301f00, classinfo=0x301f00

The 2048 case without ASLR:

sizeof(DefaultTraceInfo) == 2048
0x201f80; deref=0x1925c, deref2=0x1920c, classinfo=0x1920c
0x201f80; deref=0x1925c, deref2=0x1920c, classinfo=0x1920c

The 2052 case without ASLR:

sizeof(DefaultTraceInfo) == 2052
0x201f80; deref=0x1926c, deref2=0x1921c, classinfo=0x1921c
0x201f80; deref=0x1926c, deref2=0x1921c, classinfo=0x1921c

[kennytm]

Looks like the deeper cause is the GC. The following code path is triggered only when ASLR is enabled and the class instance size is > 2048. The fullcollect is called on the Throwable object prematurely.

#0  0x0008ecbf in uint gc.gcx.Gcx.fullcollect(void*) ()
#1  0x0008e559 in uint gc.gcx.Gcx.fullcollectshell() ()
#2  0x0008e07c in void* gc.gcx.Gcx.bigAlloc(uint, gc.gcx.Pool**, uint*) ()
#3  0x0008c41a in void* gc.gcx.GC.mallocNoSync(uint, uint, uint*) ()
#4  0x0008c281 in void* gc.gcx.GC.malloc(uint, uint, uint*) ()
#5  0x0008b684 in gc_malloc ()
#6  0x00090f63 in _d_newclass ()
#7  0x00080262 in object.Throwable.TraceInfo y.defaultTraceHandler2(void*) ()
#8  0x00082332 in _d_traceContext ()
#9  0x00090596 in _d_createTrace ()
#10 0x000901bb in _d_throwc ()
#11 0x0008018c in _Dmain ()
#12 0x00090d63 in extern (C) int rt.dmain2.main(int, char**).void runMain() ()
#13 0x0009090d in extern (C) int rt.dmain2.main(int, char**).void tryExec(scope void delegate()) ()
#14 0x00090dab in extern (C) int rt.dmain2.main(int, char**).void runAll() ()
#15 0x0009090d in extern (C) int rt.dmain2.main(int, char**).void tryExec(scope void delegate()) ()
#16 0x000908a7 in main ()

[kennytm]

Superseded by pull #43.