Real segmentation fault handler

[vstinner]

BPO	3999
Nosy	@smontanaro, @amauryfa, @abalkin, @pitrou, @vstinner
Files	segfault-3.patch: Raise MemoryError on SIGSEGV and ArithmeticError on SIGFPE fault.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2009-11-09.01:43:24.587>
created_at = <Date 2008-09-30.01:10:11.318>
labels = []
title = 'Real segmentation fault handler'
updated_at = <Date 2010-05-31.19:16:32.629>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2010-05-31.19:16:32.629>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2009-11-09.01:43:24.587>
closer = 'vstinner'
components = []
creation = <Date 2008-09-30.01:10:11.318>
creator = 'vstinner'
dependencies = []
files = ['12313', '12322']
hgrepos = []
issue_num = 3999
keywords = ['patch']
message_count = 12.0
messages = ['74060', '74066', '74067', '74068', '74089', '77478', '77479', '77563', '78745', '95055', '95095', '106804']
nosy_count = 7.0
nosy_names = ['skip.montanaro', 'amaury.forgeotdarc', 'belopolsky', 'ggenellina', 'Rhamphoryncus', 'pitrou', 'vstinner']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue3999'
versions = []

[vstinner]

I would like to be able to catch SIGSEGV in my Python code! So I
started to hack Python trunk to support this feature. The idea is to
use a signal handler which call longjmp(), and add setjmp() at
Py_EvalFrameEx() enter.

See attached ("small") patch: segfault.patch

Example read.py with the *evil* ctypes module of invalid memory read:
------------------- 8< --------------

from ctypes import string_at

def fault():
    text = string_at(1, 10)
    print("text = {0!r}".format(text))

def test():
    print("test: 1")
    try:
        fault()
    except MemoryError, err:
        print "ooops!"
        print err

    print("test: 2")
    try:
        fault()
    except MemoryError, err:
        print "ooops!"
        print err

    print("test: end")

def main():
    test()

if __name__ == "__main__":
    main()
------------------- 8< 

---

Result:
------------------- 8< --------------

$ python read.py
test: 1
sizeof()=160
ooops!
segmentation fault
test: 2
sizeof()=160
ooops!
segmentation fault
test: end
------------------- 8< 

---

Example bug1.py of a stack overflow:
----------

loop = None,
for i in xrange(10**5):
    loop = loop, None

---

Result:
----------

$ python -i bug1.py

(((((((((...Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError: segmentation fault

---

Python is able to restore a valid state (stack/heap) after a
segmentation fault and raise a classical Python exception (I choosed
MemoryError, but it could be a specific exception).

On my computer (Ubuntu Gutsy/i386), each segfault_frame takes
sizeof(sigjmpbuf) + sizeof(void*) = 160 bytes, allocated on the stack.
I don't know if it's huge or not, but that will limit the number of
recursive calls. The feature can be optional if we add a configure
option and some #ifdef/#endif. A dedicated stack is needed to be call
the signal handler on stack overflow error. I choosed 4 KB, but since
I only call longjmp(), smaller stack might also works.

Does other VM support such feature? JVM, Mono, .NET, etc. ?

I had the idea of catching SIGSEGV after reading the bpo-1069092
(stack overflow because of too many recursive calls).

[amauryfa]

Did you consider using PyOS_CheckStack for this?
Currently there is only a Windows implementation, but it seems that the
primitives you use in your patch could form a Unix version.

[vstinner]

@amaury.forgeotdarc: It looks like PyOS_CheckStack() is only
implemented for Windows. It uses alloca() + __try/__except +
_resetstkoflw(). The GNU libc nor Linux kernel don't check stack
pointer on alloca(), it's just $esp += <alloca argument>. Using
alloca() you may also be able to able outside the stack to move your
stack pointer to the heap or another memory mapping. PyOS_CheckStack()
doesn't really protect the stack: if a function use alloca() or a
similar construction like « void test(int size) { char
allocated_on_the_stack[size]; ... } », you will not catch this error.

PyOS_CheckStack() only checks one type of error: stack overflow. It
doesn't check invalid memory read / write (see my first example,
read.py).

[vstinner]

Note: my patch can be adapted to catch SIGFPE (divison by zero or
other math error). For int/long types, Python avoids divison by zero,
but for code written in C ("external modules"), Python is unable to
catch such errors. Eg. see last imageop issue: i was possible to
generate many divisions by zero.

[vstinner]

Oops, my patch was broken. I forgot to install the fault handler! Here
is a new version of the patch which also catch SIGFPE: raise an
ArithmeticError.

[vstinner]

New patch:

• limit memory footprint: use a static buffer to store the frames,
  with a maximum of MAXDEPTH frames (default: MAXDEPTH=100)
• if there are more than MAXDEPTH frames, jump to the frame MAXDEPTH
  on error (it's like a truncated traceback)
• don't call segfault_exit() in PyEval_EvalFrameEx() if
  segfault_enter() was not called

On Ubuntu Gutsy, for MAXDEPTH=100 the memory footprint is 19996 bytes.
I think that it's small, but it's possible to reduce the footprint by
using less frames or disable the alternative stack (needed for stack
overflow errors).

[vstinner]

Oh, another change in segfault-3.patch:

• disable signal handler before the first call to segfault_enter()
  and the last call to segfault_exit()

About the memory footprint: it would be possible to use variable size
buffer using malloc() and then realloc(). But static buffers are
easier to use, and I don't want to play with malloc() while I'm
handling a segmentation fault or stack overflow :-)

[vstinner]

fault.py: catch two segfaults in Python and C contexts.

[pitrou]

As mentioned in python-dev, the patch would be more suitable for
inclusion if it was changed to simply print a stack trace and bail out,
rather than try to resume execution of the Python program.

[vstinner]

My idea was rejected on python-dev mailing list. But I'm unable to
write a patch to dump a backtrace on segfault. Anyway it would be a
complelty different patch (and so a different issue). So I prefer to
close this (old) issue.

[Rhamphoryncus]

That's fine, but please provide a link to the new issue once you create it.

[vstinner]

That's fine, but please provide a link to the new issue once you create it.

Done: issue bpo-8863.