segfault caused by nested calls to map()

[dlukes]

BPO	24606
Nosy	@serhiy-storchaka, @dlukes
Superseder	bpo-14010: deeply nested itertools objects segfault

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-07-11.04:50:56.824>
created_at = <Date 2015-07-10.15:50:22.182>
labels = ['type-crash']
title = 'segfault caused by nested calls to map()'
updated_at = <Date 2015-07-11.04:50:56.794>
user = 'https://github.com/dlukes'

bugs.python.org fields:

activity = <Date 2015-07-11.04:50:56.794>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2015-07-11.04:50:56.824>
closer = 'serhiy.storchaka'
components = []
creation = <Date 2015-07-10.15:50:22.182>
creator = 'dlukes'
dependencies = []
files = []
hgrepos = []
issue_num = 24606
keywords = []
message_count = 4.0
messages = ['246567', '246588', '246591', '246593']
nosy_count = 3.0
nosy_names = ['BreamoreBoy', 'serhiy.storchaka', 'dlukes']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '14010'
type = 'crash'
url = 'https://bugs.python.org/issue24606'
versions = ['Python 3.4']

[dlukes]

The following program makes Python 3.4.3 crash with a segmentation fault:

#!/usr/bin/env python3

import operator

N = 500000
l = [0]

for i in range(N):
    l = map(operator.add, l, [1])

print(list(l))

I suppose the problem is that there are too many nested lazy calls to map, which cause a segfault when evaluated. I've played with N and surprisingly, the threshold to cause the crash varied slightly (between 130900 and 131000 on my machine).

I know that a list-comprehension, which is evaluated straight away, would be much more idiomatic for repeated element-wise addition (or numpy arrays for that matter, if available). I'm **not advocating this piece of code**, just wondering whether there couldn't be a more informative way to make Python bail out instead of the segfault? (In my real application, it took me a while to figure where the problem was without a stack trace.)

[BreamoreBoy]

FTR I can reproduce this on Windows 8.1 with 3.4.3 and 3.3.5 but not 2.7.10 or 2.6.6.

[ned-deily]

Process 51270 launched: './python' (x86_64)
Process 51270 stopped

• thread Support "bpo-" in Misc/NEWS #1: tid = 0x5c8677, 0x00000001000c1af8 python_PyObject_Alloc(use_calloc=0, ctx=<unavailable>, nelem=<unavailable>, elsize=<unavailable>) + 24 at obmalloc.c:1170, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=2, address=0x7fff5ebffff8) frame #0: 0x00000001000c1af8 python_PyObject_Alloc(use_calloc=0, ctx=, nelem=, elsize=) + 24 at obmalloc.c:1170
  1167
  1168 static void *
  1169 _PyObject_Alloc(int use_calloc, void *ctx, size_t nelem, size_t elsize)
  -> 1170 {
  1171 size_t nbytes;
  1172 block *bp;
  1173 poolp pool;
  (lldb) bt
• thread Support "bpo-" in Misc/NEWS #1: tid = 0x5c8677, 0x00000001000c1af8 python`_PyObject_Alloc(use_calloc=0, ctx=, nelem=, elsize=) + 24 at obmalloc.c:1170, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=2, address=0x7fff5ebffff8)
  • frame #0: 0x00000001000c1af8 python_PyObject_Alloc(use_calloc=0, ctx=<unavailable>, nelem=<unavailable>, elsize=<unavailable>) + 24 at obmalloc.c:1170 frame #1: 0x00000001000c0ec8 python_PyObject_Malloc(ctx=0x0000000000000000, nbytes=112) + 40 at obmalloc.c:1386
    frame Rename README to README.rst and enhance formatting #2: 0x00000001000c03e9 python_PyMem_DebugAlloc(use_calloc=0, ctx=0x0000000100390898, nbytes=80) + 153 at obmalloc.c:1838 frame #3: 0x00000001000be981 python_PyMem_DebugMalloc(ctx=0x0000000100390898, nbytes=80) + 33 at obmalloc.c:1861
    frame Update Python Software Foundation Copyright Year. #4: 0x00000001000bf3c1 pythonPyObject_Malloc(size=80) + 65 at obmalloc.c:386 frame #5: 0x000000010028c87e python_PyObject_GC_Alloc(use_calloc=0, basicsize=56) + 110 at gcmodule.c:1696
    frame Add Pycharm's .idea directory to gitignore #6: 0x000000010028c809 python_PyObject_GC_Malloc(basicsize=56) + 25 at gcmodule.c:1718 frame #7: 0x000000010028ca6d python_PyObject_GC_NewVar(tp=0x0000000100393990, nitems=2) + 109 at gcmodule.c:1747
    frame Change some mercurial/ hg.python.org references. #8: 0x00000001000d6c82 pythonPyTuple_New(size=2) + 338 at tupleobject.c:104 frame #9: 0x00000001001d7e26 pythonmap_next(lz=0x000000010cebbae8) + 38 at bltinmodule.c:1162
    frame bpo-29474: Improve documentation for weakref.WeakValueDictionary #10: 0x000000010001072c pythonPyIter_Next(iter=0x000000010cebbae8) + 44 at abstract.c:2760 frame #11: 0x00000001001d7e71 pythonmap_next(lz=0x000000010cebbbb8) + 113 at bltinmodule.c:1167
    frame bpo-29524: Add Objects/call.c file #12: 0x000000010001072c pythonPyIter_Next(iter=0x000000010cebbbb8) + 44 at abstract.c:2760 frame #13: 0x00000001001d7e71 pythonmap_next(lz=0x000000010cebbc88) + 113 at bltinmodule.c:1167
    frame Disable Travis docs job until a fix is found #14: 0x000000010001072c pythonPyIter_Next(iter=0x000000010cebbc88) + 44 at abstract.c:2760 frame #15: 0x00000001001d7e71 pythonmap_next(lz=0x000000010cebbd58) + 113 at bltinmodule.c:1167
    frame Make Travis docs build more lenient #16: 0x000000010001072c pythonPyIter_Next(iter=0x000000010cebbd58) + 44 at abstract.c:2760 frame #17: 0x00000001001d7e71 pythonmap_next(lz=0x000000010cebbe28) + 113 at bltinmodule.c:1167
    frame Rename Doc/README.txt to Doc/README.rst #18: 0x000000010001072c pythonPyIter_Next(iter=0x000000010cebbe28) + 44 at abstract.c:2760 frame #19: 0x00000001001d7e71 pythonmap_next(lz=0x000000010cebbef8) + 113 at bltinmodule.c:1167
    frame Update link destination of the Mersenne Twister homepage #20: 0x000000010001072c pythonPyIter_Next(iter=0x000000010cebbef8) + 44 at abstract.c:2760 frame #21: 0x00000001001d7e71 pythonmap_next(lz=0x000000010cebd058) + 113 at bltinmodule.c:1167
    frame [backport to 3.6] bpo-29474: Improve documentation for weakref.WeakValueDictionary #22: 0x000000010001072c pythonPyIter_Next(iter=0x000000010cebd058) + 44 at abstract.c:2760 frame #23: 0x00000001001d7e71 pythonmap_next(lz=0x000000010cebd128) + 113 at bltinmodule.c:1167
    frame bpo-28837: Fix lib2to3 handling of map/zip/filter #24: 0x000000010001072c python`PyIter_Next(iter=0x000000010cebd128) + 44 at abstract.c:2760
    [...]

[serhiy-storchaka]

This looks as a duplicate of bpo-14010.