Crashes python interpreter in seq* with simple arguments

[sandrotosi]

Hello,
i'm forwarding Debian bug #597609:

$ gdb python
GNU gdb (GDB) 7.6 (Debian 7.6-5)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/bin/python2.7...Reading symbols from /usr/lib/debug/usr/bin/python2.7...done.
done.
(gdb) run
Starting program: /usr/bin/python2.7 
warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7ffff7ffa000
warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Python 2.7.5+ (default, Sep 17 2013, 15:31:50) 
[GCC 4.8.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import Levenshtein
>>> Levenshtein.seqratio("hallo", "bla")

Program received signal SIGSEGV, Segmentation fault.
extract_stringlist (list='hallo', name=name@entry=0x7ffff64b2c00 "seqratio", n=n@entry=5, sizelist=sizelist@entry=0x7fffffffdd90, strlist=strlist@entry=0x7fffffffdd80) at Levenshtein.c:1166
1166    Levenshtein.c: No such file or directory.
(gdb) bt full
#0  extract_stringlist (list='hallo', name=name@entry=0x7ffff64b2c00 "seqratio", n=n@entry=5, sizelist=sizelist@entry=0x7fffffffdd90, strlist=strlist@entry=0x7fffffffdd80)
    at Levenshtein.c:1166
        i = <optimized out>
        first = <unknown at remote 0x-2bd86a95779d78df>
#1  0x00007ffff64ac59e in setseq_common (args=<optimized out>, name=name@entry=0x7ffff64b2c00 "seqratio", foo=..., lensum=lensum@entry=0x7fffffffddf8) at Levenshtein.c:1319
        n1 = 5
        n2 = 3
        strings1 = 0x0
        strings2 = 0x0
        sizes1 = 0x0
        sizes2 = 0x0
        strlist1 = 'hallo'
        strlist2 = 'bla'
        strseq1 = <optimized out>
        strseq2 = ['b', 'l', 'a']
        stringtype1 = <optimized out>
        stringtype2 = <optimized out>
        r = -1
#2  0x00007ffff64b0676 in seqratio_py (self=<optimized out>, args=<optimized out>) at Levenshtein.c:1251
        lensum = 8
        r = <optimized out>
#3  0x0000000000529e45 in call_function (oparg=<optimized out>, pp_stack=0x7fffffffdf00) at ../Python/ceval.c:4021
        flags = <optimized out>
        tstate = 0x9410a0
        func = <built-in function seqratio>
        w = <optimized out>
        na = <optimized out>
        nk = <optimized out>
        n = <optimized out>
        pfunc = 0xa2ed08
        x = <optimized out>
#4  PyEval_EvalFrameEx (f=f@entry=Frame 0xa2eb90, for file <stdin>, line 1, in <module> (), throwflag=throwflag@entry=0) at ../Python/ceval.c:2666
        sp = 0xa2ed10
        stack_pointer = <optimized out>
        next_instr = 0x7ffff7ee5763 "Fd\002"
        opcode = <optimized out>
        oparg = <optimized out>
        why = WHY_NOT
        err = <optimized out>
        x = <optimized out>
        v = <optimized out>
        w = <optimized out>
        u = <optimized out>
---Type <return> to continue, or q <return> to quit---
        t = <optimized out>
        stream = 0x0
        fastlocals = 0xa2ed08
        freevars = <optimized out>
        retval = <optimized out>
        tstate = <optimized out>
        co = <optimized out>
        instr_ub = -1
        instr_lb = 0
        instr_prev = -1
        first_instr = <optimized out>
        names = <optimized out>
        consts = <optimized out>
        enter = '__enter__'
        exit = '__exit__'
#5  0x00000000004c6544 in PyEval_EvalCodeEx (closure=0x0, defcount=0, defs=0x0, kwcount=0, kws=0x0, argcount=0, args=0x0, locals=<optimized out>, globals=<optimized out>, co=0x7ffff7f371b0)
    at ../Python/ceval.c:3253
        retval = 0x0
        fastlocals = 0xa2ed08
        freevars = 0xa2ed08
        u = <optimized out>
        f = Frame 0xa2eb90, for file <stdin>, line 1, in <module> ()
        tstate = 0x9410a0
        x = <optimized out>
#6  PyEval_EvalCode (locals=<optimized out>, globals=<optimized out>, co=0x7ffff7f371b0) at ../Python/ceval.c:667
No locals.
#7  run_mod.42568 (mod=mod@entry=0xa2cce0, filename=filename@entry=0x5bb2b5 "<stdin>", globals=<optimized out>, locals=<optimized out>, flags=flags@entry=0x7fffffffe0c0, 
    arena=arena@entry=0x9ab2d0) at ../Python/pythonrun.c:1365
        co = 0x7ffff7f371b0
#8  0x000000000043407e in PyRun_InteractiveOneFlags (fp=fp@entry=0x7ffff729d240 <_IO_2_1_stdin_>, filename=filename@entry=0x5bb2b5 "<stdin>", flags=flags@entry=0x7fffffffe0c0)
    at ../Python/pythonrun.c:852
        m = <optimized out>
        d = <optimized out>
        v = '>>> '
        w = '... '
        mod = 0xa2cce0
        arena = 0x9ab2d0
        ps1 = <optimized out>
        ps2 = 0x7ffff7eed3b4 "... "
        errcode = 0
#9  0x000000000043419a in PyRun_InteractiveLoopFlags (fp=fp@entry=0x7ffff729d240 <_IO_2_1_stdin_>, filename=filename@entry=0x5bb2b5 "<stdin>", flags=flags@entry=0x7fffffffe0c0)
    at ../Python/pythonrun.c:772
        v = <optimized out>
---Type <return> to continue, or q <return> to quit---
        ret = <optimized out>
        local_flags = {cf_flags = 0}
#10 0x000000000043484f in PyRun_AnyFileExFlags (fp=fp@entry=0x7ffff729d240 <_IO_2_1_stdin_>, filename=filename@entry=0x5bb2b5 "<stdin>", closeit=closeit@entry=0, 
    flags=flags@entry=0x7fffffffe0c0) at ../Python/pythonrun.c:741
        err = <optimized out>
#11 0x00000000004353e3 in Py_Main (argc=<optimized out>, argv=0x7fffffffe278) at ../Modules/main.c:640
        c = <optimized out>
        sts = <optimized out>
        command = 0x0
        filename = 0x0
        module = 0x0
        fp = 0x7ffff729d240 <_IO_2_1_stdin_>
        p = <optimized out>
        unbuffered = <optimized out>
        skipfirstline = <optimized out>
        stdin_is_interactive = 1
        help = <optimized out>
        version = <optimized out>
        saw_unbuffered_flag = <optimized out>
        cf = {cf_flags = 0}
#12 0x00007ffff6f17995 in __libc_start_main (main=0x4354a1 <main>, argc=1, ubp_av=0x7fffffffe278, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, 
    stack_end=0x7fffffffe268) at libc-start.c:260
        result = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {0, -2830849066025673389, 5720549, 140737488347760, 0, 0, 2830849065964215635, 2830829193398394195}, mask_was_saved = 0}}, priv = {pad = {
              0x0, 0x0, 0x5b8e60 <__libc_csu_init>, 0x7fffffffe278}, data = {prev = 0x0, cleanup = 0x0, canceltype = 6000224}}}
        not_first_call = <optimized out>
#13 0x0000000000574a0e in _start ()
No symbol table info available.
(gdb) thread apply all backtrace

Thread 1 (Thread 0x7ffff7fc1700 (LWP 8108)):
#0  extract_stringlist (list='hallo', name=name@entry=0x7ffff64b2c00 "seqratio", n=n@entry=5, sizelist=sizelist@entry=0x7fffffffdd90, strlist=strlist@entry=0x7fffffffdd80)
    at Levenshtein.c:1166
#1  0x00007ffff64ac59e in setseq_common (args=<optimized out>, name=name@entry=0x7ffff64b2c00 "seqratio", foo=..., lensum=lensum@entry=0x7fffffffddf8) at Levenshtein.c:1319
#2  0x00007ffff64b0676 in seqratio_py (self=<optimized out>, args=<optimized out>) at Levenshtein.c:1251
#3  0x0000000000529e45 in call_function (oparg=<optimized out>, pp_stack=0x7fffffffdf00) at ../Python/ceval.c:4021
#4  PyEval_EvalFrameEx (f=f@entry=Frame 0xa2eb90, for file <stdin>, line 1, in <module> (), throwflag=throwflag@entry=0) at ../Python/ceval.c:2666
#5  0x00000000004c6544 in PyEval_EvalCodeEx (closure=0x0, defcount=0, defs=0x0, kwcount=0, kws=0x0, argcount=0, args=0x0, locals=<optimized out>, globals=<optimized out>, co=0x7ffff7f371b0)
    at ../Python/ceval.c:3253
#6  PyEval_EvalCode (locals=<optimized out>, globals=<optimized out>, co=0x7ffff7f371b0) at ../Python/ceval.c:667
#7  run_mod.42568 (mod=mod@entry=0xa2cce0, filename=filename@entry=0x5bb2b5 "<stdin>", globals=<optimized out>, locals=<optimized out>, flags=flags@entry=0x7fffffffe0c0, 
    arena=arena@entry=0x9ab2d0) at ../Python/pythonrun.c:1365
#8  0x000000000043407e in PyRun_InteractiveOneFlags (fp=fp@entry=0x7ffff729d240 <_IO_2_1_stdin_>, filename=filename@entry=0x5bb2b5 "<stdin>", flags=flags@entry=0x7fffffffe0c0)
    at ../Python/pythonrun.c:852
#9  0x000000000043419a in PyRun_InteractiveLoopFlags (fp=fp@entry=0x7ffff729d240 <_IO_2_1_stdin_>, filename=filename@entry=0x5bb2b5 "<stdin>", flags=flags@entry=0x7fffffffe0c0)
    at ../Python/pythonrun.c:772
#10 0x000000000043484f in PyRun_AnyFileExFlags (fp=fp@entry=0x7ffff729d240 <_IO_2_1_stdin_>, filename=filename@entry=0x5bb2b5 "<stdin>", closeit=closeit@entry=0, 
    flags=flags@entry=0x7fffffffe0c0) at ../Python/pythonrun.c:741
#11 0x00000000004353e3 in Py_Main (argc=<optimized out>, argv=0x7fffffffe278) at ../Modules/main.c:640
#12 0x00007ffff6f17995 in __libc_start_main (main=0x4354a1 <main>, argc=1, ubp_av=0x7fffffffe278, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, 
    stack_end=0x7fffffffe268) at libc-start.c:260
#13 0x0000000000574a0e in _start ()
(gdb) quit
A debugging session is active.

        Inferior 1 [process 8108] will be killed.

Quit anyway? (y or n) y

[metaljoe]

I have the same problem on OS X:

Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49) 
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import Levenshtein
>>> Levenshtein.seqratio("apples", "oranges")
Segmentation fault

The traceback is:

Process:         Python [11538]
Path:            /System/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python
Identifier:      Python
Version:         ??? (???)
Code Type:       X86-64 (Native)
Parent Process:  bash [806]

PlugIn Path:       /Library/Python/2.6/site-packages/python_Levenshtein-0.11.2-py2.6-macosx-10.6-universal.egg/Levenshtein.so
PlugIn Identifier: Levenshtein.so
PlugIn Version:    ??? (???)

Date/Time:       2014-08-18 17:00:53.560 +0100
OS Version:      Mac OS X 10.6.8 (10K549)
Report Version:  6

Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: 0x000000000000000d, 0x0000000000000000
Crashed Thread:  0  Dispatch queue: com.apple.main-thread

Thread 0 Crashed:  Dispatch queue: com.apple.main-thread
0   Levenshtein.so                  0x00000001001e26ea extract_stringlist + 103
1   Levenshtein.so                  0x00000001001e2ca9 setseq_common + 505
2   Levenshtein.so                  0x00000001001e2efa seqratio_py + 59
3   org.python.python               0x0000000100089187 PyEval_EvalFrameEx + 15317
4   org.python.python               0x000000010008acce PyEval_EvalCodeEx + 1803
5   org.python.python               0x000000010008ad61 PyEval_EvalCode + 54
6   org.python.python               0x00000001000a265a Py_CompileString + 78
7   org.python.python               0x00000001000a44dd PyRun_InteractiveOneFlags + 503
8   org.python.python               0x00000001000a4615 PyRun_InteractiveLoopFlags + 206
9   org.python.python               0x00000001000a4685 PyRun_AnyFileExFlags + 76
10  org.python.python               0x00000001000b0286 Py_Main + 2718
11  org.python.python.app           0x0000000100000e6c start + 52

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000100124ce0  rbx: 0x8447ec0077b42b75  rcx: 0x00007fff5fbff2f8  rdx: 0x0000000000000006
  rdi: 0x000000010054b7e0  rsi: 0x00000001001e9324  rbp: 0x00007fff5fbff2a0  rsp: 0x00007fff5fbff250
   r8: 0x00007fff5fbff308   r9: 0x00000001002fc0a4  r10: 0x000000000000001b  r11: 0x000000010003dd21
  r12: 0x000000010047cd40  r13: 0x000000010054b7e0  r14: 0x0000000100544ef0  r15: 0x00000001001e3b20
  rip: 0x00000001001e26ea  rfl: 0x0000000000010293  cr2: 0x00000001001e2ebf

Binary Images:
       0x100000000 -        0x100000ff7  org.python.python.app 2.6 (2.6) <3F4A329D-3EF9-A736-6F3D-67A4B39AE8DE> /System/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python
       0x100004000 -        0x100114ff7  org.python.python 2.6.1 (2.6.1) <126DA8FF-5BC2-8788-51E3-D7A29A3F9F0F> /System/Library/Frameworks/Python.framework/Versions/2.6/Python
       0x1001d8000 -        0x1001daff7  readline.so ??? (???) <200F07EB-D7AC-BE0E-165E-A84266ECC26F> /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload/readline.so
       0x1001e0000 -        0x1001ecfff +Levenshtein.so ??? (???) <C15F5478-3A82-6CB5-6001-CDA897477489> /Library/Python/2.6/site-packages/python_Levenshtein-0.11.2-py2.6-macosx-10.6-universal.egg/Levenshtein.so
       0x100570000 -        0x10058cfe7  libedit.2.dylib 2.11.0 (compatibility 2.0.0) <8DC2232B-8AE0-E83B-1639-A78D00F33E3E> /usr/lib/libedit.2.dylib
    0x7fff5fc00000 -     0x7fff5fc3bdef  dyld 132.1 (???) <B536F2F1-9DF1-3B6C-1C2C-9075EA219A06> /usr/lib/dyld
    0x7fff8011e000 -     0x7fff80122ff7  libmathCommon.A.dylib 315.0.0 (compatibility 1.0.0) <95718673-FEEE-B6ED-B127-BCDBDB60D4E5> /usr/lib/system/libmathCommon.A.dylib
    0x7fff828d0000 -     0x7fff82986ff7  libobjc.A.dylib 227.0.0 (compatibility 1.0.0) <03140531-3B2D-1EBA-DA7F-E12CC8F63969> /usr/lib/libobjc.A.dylib
    0x7fff82a71000 -     0x7fff82a82ff7  libz.1.dylib 1.2.3 (compatibility 1.0.0) <FB5EE53A-0534-0FFA-B2ED-486609433717> /usr/lib/libz.1.dylib
    0x7fff86544000 -     0x7fff865c1fef  libstdc++.6.dylib 7.9.0 (compatibility 7.0.0) <35ECA411-2C08-FD7D-11B1-1B7A04921A5C> /usr/lib/libstdc++.6.dylib
    0x7fff86c2f000 -     0x7fff86df0fef  libSystem.B.dylib 125.2.11 (compatibility 1.0.0) <9AB4F1D1-89DC-0E8A-DC8E-A4FE4D69DB69> /usr/lib/libSystem.B.dylib
    0x7fff87633000 -     0x7fff877f1fff  libicucore.A.dylib 40.0.0 (compatibility 1.0.0) <97A75BFB-0DB6-6F44-36B0-97B7F7208ABB> /usr/lib/libicucore.A.dylib
    0x7fff87c03000 -     0x7fff87d7afe7  com.apple.CoreFoundation 6.6.6 (550.44) <BB4E5158-E47A-39D3-2561-96CB49FA82D4> /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
    0x7fff89688000 -     0x7fff896c7fef  libncurses.5.4.dylib 5.4.0 (compatibility 5.4.0) <E1F34D53-3D62-78C0-CAD8-8AD22C110A9E> /usr/lib/libncurses.5.4.dylib
    0x7fff8a78b000 -     0x7fff8a7d7fff  libauto.dylib ??? (???) <F7221B46-DC4F-3153-CE61-7F52C8C293CF> /usr/lib/libauto.dylib
    0x7fffffe00000 -     0x7fffffe01fff  libSystem.B.dylib ??? (???) <9AB4F1D1-89DC-0E8A-DC8E-A4FE4D69DB69> /usr/lib/libSystem.B.dylib

Not sure if it's relevant or not, but the C code doesn't compile cleanly on installation as I see a lot of warnings like "Levenshtein.c:694: warning: pointer targets in assignment differ in signedness".

[ztane]

Indeed, the seqratio should not accept bare strings.

[ztane]

It turns out there was always a critical bug since the beginning of Git history, that I now fixed.

Namely, the code correctly used PySequence_Fast to create a new list from original sequence, but then reverted using the original instead of the returned object. This worked for the basic uses of the method, where the 2 arguments would have been lists or tuples; as in those case PySequence_Fast would have returned an incremented reference to the original object.