Print stack overflow messages for Linux and OS X

[Zoxc]

This installs signal handlers to print out stack overflow messages on Linux. It also ensures the main thread has a guard page.

This will catch stack overflows in external code. It's done in preparation of switching to stack probes (#16012).

I've done some simple tests with overflowing the main thread, native threads and green threads (with and without UV) on x86-64.
This might work on ARM, MIPS and x86-32.

I've been unable to run the test suite on this because of #16305.

[huonw]

Is there a particular reason this can't be in Rust?

[alexcrichton]

Yes, I believe this should all largely be written in rust rather than in C (other signal handlers are already written in rust). I'm also hesitant to land linux-only support for this feature. I would much rather implement everything in one sweep than be in a confusing state for awhile while we're waiting on the next few patches.

[Zoxc]

I wrote it in C since I don't find bindings for signal things. I see there's some in libnative (why are those there?), but that's still missing sigaltstack, SIGSTKSZ, stack_t, siginfo_t.si_addr.

Isn't rust-bindgen good enough to handle system headers?

[huonw]

You can easily write the necessary bindings (or extend the ones that exist already), e.g.

struct stack_t {
    ss_sp: *mut libc::c_void,
    ss_flags: libc::c_int,
    ss_size: libc::size_t    
}
extern {
    fn sigaltstack(ss: *const stack_t, oss: *mut stack_t) -> libc::c_int
}

Isn't rust-bindgen good enough to handle system headers?

The in-tree extern bindings are hand-written, not created with rust-bindgen.

[pcwalton]

I'm fine with making this Linux-only for now, as I prefer things to be in
tree when possible to reduce rebase pain.
On Aug 9, 2014 8:50 PM, "Alex Crichton" notifications@github.com wrote:

Yes, I believe this should all largely be written in rust rather than in C
(other signal handlers are already written in rust). I'm also hesitant to
land linux-only support for this feature. I would much rather implement
everything in one sweep than be in a confusing state for awhile while we're
waiting on the next few patches.

—
Reply to this email directly or view it on GitHub
#16388 (comment).

[Zoxc]

This works on Linux and OS X now.

[alexcrichton]

My system (OSX 10.9) has this definition:

struct  __sigaction {                                                 
        union __sigaction_u __sigaction_u;  /* signal handler */      
        void    (*sa_tramp)(void *, int, int, siginfo_t *, void *);   
        sigset_t sa_mask;               /* signal mask to apply */    
        int     sa_flags;               /* see signal options below */
};                                                                    

How come you changed it?

[Zoxc]

That's __sigaction not sigaction.

[alexcrichton]

Aha! I should remind myself that I have eyes! I suspect that ios is the same in this regard in that case and the upper one should probably just be deleted.

[alexcrichton]

I'll try to take a closer look at this soon (sadly not before next week), but how much of an effort do you think this will be to extend this strategy to windows? I would hate to commit to a unix-only design only to find out later that it isn't compatible with windows for one reason or another. You may also want to download the arm/android toolchain as you should be able to just tweak a static here or there to get support. If you have a list of missing statics as well, I can look those up for you.

[Zoxc]

This should be simpler on Windows (with the exception of green threads which would need the same handling as Unix). However it Windows does not install guard pages on threads created by CreateThread, nor will it call the vectored exception handler I installed. I think we're doing something bad in the runtime somewhere (excluding writing to fs:0x14).

[alexcrichton]

I'd rather not add a new public function for this, and this seems like something that callers of new need to worry about, so could you add an Option<uint> parameter to new instead?

[alexcrichton]

Also, can you add tests for all the functionality added here? You should be able to just spawn a subprocess and then inspect the output to ensure that the handlers were triggered correctly. Some tests that should be written:

• Native overflow in C code
• Green overflow in C code
• Native segfault (not an overflow)
• Green segfault (not an overflow)

You can ignore the test for unimplemented platforms for now.

[alexcrichton]

Another idea I thought of today is that you may be able to remove a good portion of the guard page logic by using stack::get_sp_limit() + stack::RED_ZONE for testing whether a fault is near a guard page or not.

[Zoxc]

This passes tests on try now (with the exception of BSD, which I hopefully fixed)

[Zoxc]

I see you test for Android too, so I added (untested) support for it.

[Zoxc]

All Android test fails with error: undefined reference to 'signal'. Any idea what's missing?

[Zoxc]

I'm just disabling Android for now. We can fix that later.

[alexcrichton]

Sure, we can land this for now with android disabled. Could you open an issue on enabling for android @Zoxc?

FYI this is the definition in the header file:

extern __sighandler_t bsd_signal(int, __sighandler_t);          

/* the default is bsd */                                        
static __inline__ __sighandler_t signal(int s, __sighandler_t f)
{                                                               
    return bsd_signal(s,f);                                     
}