Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hashlib segmentation fault #79103

Closed
httpsgithubcomxcainiao mannequin opened this issue Oct 7, 2018 · 16 comments
Closed

hashlib segmentation fault #79103

httpsgithubcomxcainiao mannequin opened this issue Oct 7, 2018 · 16 comments
Labels
3.7 (EOL) end of life 3.8 (EOL) end of life extension-modules C modules in the Modules dir type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@httpsgithubcomxcainiao
Copy link
Mannequin

httpsgithubcomxcainiao mannequin commented Oct 7, 2018

BPO 34922
Nosy @terryjreedy, @vstinner, @tiran, @ned-deily, @serhiy-storchaka, @tirkarthi, @https://github.com/xcainiao
PRs
  • bpo-34922: Fixed integer overflow in the digest() and hexdigest() methods #9751
  • [3.6] bpo-34922: Fix integer overflow in the digest() and hexdigest() methods (GH-9751) #9797
  • [3.7] bpo-34922: Fix integer overflow in the digest() and hexdigest() methods (GH-9751) #9798
  • [3.6] bpo-34922: Fix integer overflow in the digest() and hexdigest() methods (GH-9751) (GH-9798) #9801
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2020-01-14.22:56:24.735>
    created_at = <Date 2018-10-07.12:40:29.942>
    labels = ['extension-modules', '3.7', '3.8', 'type-crash']
    title = 'hashlib segmentation fault'
    updated_at = <Date 2020-01-14.22:56:24.734>
    user = 'https://github.com/httpsgithubcomxcainiao'

    bugs.python.org fields:

    activity = <Date 2020-01-14.22:56:24.734>
    actor = 'ned.deily'
    assignee = 'none'
    closed = True
    closed_date = <Date 2020-01-14.22:56:24.735>
    closer = 'ned.deily'
    components = ['Extension Modules']
    creation = <Date 2018-10-07.12:40:29.942>
    creator = 'shuoz'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 34922
    keywords = ['patch']
    message_count = 16.0
    messages = ['327277', '327283', '327285', '327288', '327294', '327306', '327312', '327495', '327515', '327518', '327519', '327604', '327606', '327619', '327677', '360009']
    nosy_count = 7.0
    nosy_names = ['terry.reedy', 'vstinner', 'christian.heimes', 'ned.deily', 'serhiy.storchaka', 'xtreak', 'shuoz']
    pr_nums = ['9751', '9797', '9798', '9801']
    priority = None
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'crash'
    url = 'https://bugs.python.org/issue34922'
    versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

    @httpsgithubcomxcainiao
    Copy link
    Mannequin Author

    httpsgithubcomxcainiao mannequin commented Oct 7, 2018

    python hashlib a signd overflow maybe cause a memory over read.

    python version:
    Python 3.6.7rc1+ (heads/3.6:cb0bec3, Oct 1 2018, 02:19:39)
    [GCC 7.3.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.

    [----------------------------------registers-----------------------------------]
    RAX: 0x0
    RBX: 0x7fffffffd5f0 --> 0x41b58ab3
    RCX: 0x0
    RDX: 0x1ffffffffffffff6
    RSI: 0x7ffff35ae880 --> 0x0
    RDI: 0x7fffffffd650 --> 0x7d828fe8a42b9c7f
    RBP: 0xffffffffabe --> 0x0
    RSP: 0x7fffffffd5c8 --> 0x7ffff2a5f793 (<_sha3_shake_128_hexdigest+627>:	test   eax,eax)
    RIP: 0x7ffff2a5ec60 (<_PySHA3_KeccakWidth1600_SpongeSqueeze>:	push   r15)
    R8 : 0x65fc7ba985946aff
    R9 : 0xefbdaa140b587a16
    R10: 0x50573373c9b2b8dc
    R11: 0xfba4d93abbdabffc
    R12: 0x7fffffffd770 --> 0x7fffffffd7d0 --> 0xffffffffb00 --> 0x0
    R13: 0x7fffffffd650 --> 0x7d828fe8a42b9c7f
    R14: 0x7ffff35ae880 --> 0x0
    R15: 0xfffffffffffffff6
    EFLAGS: 0xa06 (carry PARITY adjust zero sign trap INTERRUPT direction OVERFLOW)
    [-------------------------------------code-------------------------------------]
       0x7ffff2a5ec50 <_PySHA3_KeccakP1600_ExtractBytes+160>:	jmp    0x7ffff2a54d10 <_PySHA3_KeccakP1600_ExtractBytesInLane@plt>
       0x7ffff2a5ec55:	nop
       0x7ffff2a5ec56:	nop    WORD PTR cs:[rax+rax*1+0x0]
    => 0x7ffff2a5ec60 <_PySHA3_KeccakWidth1600_SpongeSqueeze>:	push   r15
       0x7ffff2a5ec62 <_PySHA3_KeccakWidth1600_SpongeSqueeze+2>:	push   r14
       0x7ffff2a5ec64 <_PySHA3_KeccakWidth1600_SpongeSqueeze+4>:	push   r13
       0x7ffff2a5ec66 <_PySHA3_KeccakWidth1600_SpongeSqueeze+6>:	push   r12
       0x7ffff2a5ec68 <_PySHA3_KeccakWidth1600_SpongeSqueeze+8>:	mov    r13,rdx
    [------------------------------------stack-------------------------------------]
    0000| 0x7fffffffd5c8 --> 0x7ffff2a5f793 (<_sha3_shake_128_hexdigest+627>:	test   eax,eax)
    0008| 0x7fffffffd5d0 --> 0x7fffffffd5f0 --> 0x41b58ab3
    0016| 0x7fffffffd5d8 --> 0xffffefdb33b --> 0x0
    0024| 0x7fffffffd5e0 --> 0x7ffff7ed99d8 --> 0x0
    0032| 0x7fffffffd5e8 --> 0x7ffff3606910 --> 0x6190000096e5 --> 0x9000009828000000
    0040| 0x7fffffffd5f0 --> 0x41b58ab3
    0048| 0x7fffffffd5f8 --> 0x7ffff2a68c08 ("2 32 8 6 length 96 224 4 temp ")
    0056| 0x7fffffffd600 --> 0x7ffff2a5f520 (<_sha3_shake_128_hexdigest>:	push   r15)
    [------------------------------------------------------------------------------]
    Legend: code, data, rodata, value
    
    Breakpoint 2, _PySHA3_KeccakWidth1600_SpongeSqueeze (instance=0x7fffffffd650, data=0x7ffff35ae880 "", dataByteLen=0x1ffffffffffffff6) at /home/test/cpython/Modules/_sha3/kcp/KeccakSponge.inc:272
    ``````py
    dataByteLen=0x1ffffffffffffff6
    
    
    
    RAX: 0x7ffff3615f90 --> 0xfffffffffffffffa
    RBX: 0xa8
    RCX: 0x7ffff3616028 --> 0xf938000001a4
    RDX: 0x18
    RSI: 0x7fffffffd6e0 --> 0x6ab2a5fe4fe8efd
    RDI: 0x7ffff3615fe0 --> 0x44b6a41dfdc1a3df
    RBP: 0x7fffffffd510 --> 0xa8
    RSP: 0x7fffffffcc78 --> 0x7ffff6e936cf (mov    rcx,QWORD PTR [rbp-0x38])
    RIP: 0x7ffff6120786 (<__memmove_sse2_unaligned_erms+614>:	movntdq XMMWORD PTR [rdi+0x20],xmm2)
    R8 : 0xfffffffffffffff0
    R9 : 0x10007e6bac07 --> 0x0
    R10: 0x7ffff3616038 --> 0x0
    R11: 0x7ffff3615f90 --> 0xfffffffffffffffa
    R12: 0x7ffff3615f90 --> 0xfffffffffffffffa
    R13: 0x7fffffffd650 --> 0xa35bf3e9cd13e78e
    R14: 0x7ffff3615f90 --> 0xfffffffffffffffa
    R15: 0x0
    EFLAGS: 0x10206 (carry PARITY adjust zero sign trap INTERRUPT direction overflow)
    [-------------------------------------code-------------------------------------]
       0x7ffff6120779 <__memmove_sse2_unaligned_erms+601>:	sub    rdx,0x40
       0x7ffff612077d <__memmove_sse2_unaligned_erms+605>:	movntdq XMMWORD PTR [rdi],xmm0
       0x7ffff6120781 <__memmove_sse2_unaligned_erms+609>:	movntdq XMMWORD PTR [rdi+0x10],xmm1
    => 0x7ffff6120786 <__memmove_sse2_unaligned_erms+614>:	movntdq XMMWORD PTR [rdi+0x20],xmm2
       0x7ffff612078b <__memmove_sse2_unaligned_erms+619>:	movntdq XMMWORD PTR [rdi+0x30],xmm3
       0x7ffff6120790 <__memmove_sse2_unaligned_erms+624>:	add    rdi,0x40
       0x7ffff6120794 <__memmove_sse2_unaligned_erms+628>:	cmp    rdx,0x40
       0x7ffff6120798 <__memmove_sse2_unaligned_erms+632>:	ja     0x7ffff6120758 <__memmove_sse2_unaligned_erms+568>
    [------------------------------------stack-------------------------------------]
    0000| 0x7fffffffcc78 --> 0x7ffff6e936cf (mov    rcx,QWORD PTR [rbp-0x38])
    0008| 0x7fffffffcc80 --> 0x7fffffffccf0 --> 0x41b58ab3
    0016| 0x7fffffffcc88 --> 0x7fffffffcd90 --> 0x6
    0024| 0x7fffffffcc90 --> 0xffffffff99e --> 0x0
    0032| 0x7fffffffcc98 --> 0x7fffffffcd50 --> 0x0
    0040| 0x7fffffffcca0 --> 0x0
    0048| 0x7fffffffcca8 --> 0x7ffff3616038 --> 0x0
    0056| 0x7fffffffccb0 --> 0x7ffff358a068 --> 0x1
    [------------------------------------------------------------------------------]
    Legend: code, data, rodata, value
    Stopped reason: SIGSEGV
    __memmove_sse2_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:492
    492	../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S: No such file or directory.
    gdb-peda$ bt
    #0  __memmove_sse2_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:492
    #1  0x00007ffff6e936cf in ?? () from /usr/lib/x86_64-linux-gnu/libasan.so.4
    #2  0x00007ffff2a5eab4 in memcpy (__len=0xa8, __src=<optimized out>, __dest=<optimized out>) at /usr/include/x86_64-linux-gnu/bits/string_fortified.h:34
    #3  _PySHA3_KeccakP1600_ExtractLanes (state=<optimized out>, data=<optimized out>, laneCount=0x15) at /home/test/cpython/Modules/_sha3/kcp/KeccakP-1600-opt64.c:342
    #4  0x00007ffff2a5ec2c in _PySHA3_KeccakP1600_ExtractBytes (state=0x7fffffffd650, data=0x7ffff3615f90 "\372\377\377\377\377\377\377\377\002", offset=<optimized out>, length=0xa8)
        at /home/test/cpython/Modules/_sha3/kcp/KeccakP-1600-opt64.c:375
    #5  0x00007ffff2a5ee1d in _PySHA3_KeccakWidth1600_SpongeSqueeze (instance=0x7fffffffd650, data=<optimized out>, dataByteLen=0x1ffffffffffffff6)
        at /home/test/cpython/Modules/_sha3/kcp/KeccakSponge.inc:287
    #6  0x00007ffff2a5f793 in _SHAKE_digest (hex=0x1, digestlen=0xfffffffffffffff6, self=0x7ffff7ed98e8) at /home/test/cpython/Modules/_sha3/sha3module.c:620
    #7  _sha3_shake_128_hexdigest_impl (length=0xfffffffffffffff6, self=0x7ffff7ed98e8) at /home/test/cpython/Modules/_sha3/sha3module.c:669
    #8  _sha3_shake_128_hexdigest (self=0x7ffff7ed98e8, args=<optimized out>, nargs=<optimized out>, kwnames=<optimized out>) at /home/test/cpython/Modules/_sha3/clinic/sha3module.c.h:149
    #9  0x000055555583eab6 in _PyCFunction_FastCallDict (kwargs=0x0, nargs=0x1, args=0x616000021518, func_obj=0x7ffff2e86f30) at Objects/methodobject.c:250
    #10 _PyCFunction_FastCallKeywords (func=func@entry=0x7ffff2e86f30, stack=0x616000021518, nargs=nargs@entry=0x1, kwnames=kwnames@entry=0x0) at Objects/methodobject.c:294
    #11 0x0000555555995945 in call_function (pp_stack=pp_stack@entry=0x7fffffffdc30, oparg=oparg@entry=0x1, kwnames=kwnames@entry=0x0) at Python/ceval.c:4837
    #12 0x000055555599feaa in _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3335
    #13 0x0000555555994939 in PyEval_EvalFrameEx (throwflag=0x0, f=0x616000021398) at Python/ceval.c:754
    #14 _PyEval_EvalCodeWithName (_co=_co@entry=0x7ffff36088a0, globals=globals@entry=0x0, locals=locals@entry=0x7ffff355a9d8, args=args@entry=0x0, argcount=argcount@entry=0x0, kwnames=kwnames@entry=0x0,
        kwargs=0x0, kwcount=0x0, kwstep=0x2, defs=0x0, defcount=0x0, kwdefs=0x0, closure=0x0, name=0x0, qualname=0x0) at Python/ceval.c:4166
    #15 0x0000555555997b73 in PyEval_EvalCodeEx (closure=0x0, kwdefs=0x0, defcount=0x0, defs=0x0, kwcount=0x0, kws=0x0, argcount=0x0, args=0x0, locals=locals@entry=0x7ffff355a9d8, globals=globals@entry=0x0,
        _co=_co@entry=0x7ffff36088a0) at Python/ceval.c:4187
    #16 PyEval_EvalCode (co=co@entry=0x7ffff36088a0, globals=globals@entry=0x7ffff7e5a318, locals=locals@entry=0x7ffff7e5a318) at Python/ceval.c:731
    #17 0x00005555556b5b3b in run_mod (arena=0x7ffff7e75150, flags=<optimized out>, locals=0x7ffff7e5a318, globals=0x7ffff7e5a318, filename=0x7ffff358d270, mod=0x62500001e300) at Python/pythonrun.c:1025
    #18 PyRun_FileExFlags (fp=<optimized out>, filename_str=<optimized out>, start=<optimized out>, globals=<optimized out>, locals=<optimized out>, closeit=<optimized out>, flags=<optimized out>)
        at Python/pythonrun.c:978
    #19 0x00005555556b5fdc in PyRun_SimpleFileExFlags (fp=<optimized out>,
        filename=0x7ffff35c2680 "\314\070\064\302\227\a\254\bJf\331u\230N\273\022\355@\200\352\024`z[\267&\257+\022Q\324\017\310\nSyF2+\001{\327\354\355\245\275\002\064d-\235x\\\327O\230٧\036ތF\222\326\336\060\027q\220\037\217\b\364#=\366\224,\362\355\224i4h\030.c\377\225\360.׀M\033\066\251\ve'M=\261\t\365\307\016\267\203Q\316\313n\251]+\351H\222\244\266{\224FG\257\022\340\071\233r\300\220\065\031\236][\266\v\027\071#\354Ɣ\310\\\243M\243\251\250\372_\362^Φ\306ڝ\222\365\062O1nY\224pĥ\243IV\364\070\356\232\\\222z\242\321\v\027|\342\027\325\325O֬\300\252a0\250"..., closeit=0x1, flags=<optimized out>)
        at Python/pythonrun.c:419
    #20 0x00005555556f2704 in run_file (p_cf=0x7fffffffe2b0, filename=0x604000000010 L"crash.py", fp=0x616000034880) at Modules/main.c:340
    #21 Py_Main (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:810
    #22 0x000055555569a293 in main (argc=argc@entry=0x2, argv=argv@entry=0x7fffffffe528) at ./Programs/python.c:69
    #23 0x00007ffff6086b97 in __libc_start_main (main=0x55555569a050 <main>, argc=0x2, argv=0x7fffffffe528, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe518)
        at ../csu/libc-start.c:310
    #24 0x000055555569bb2a in _start ()
    

    x.py

    import hashlib
    hashlib.shake_128().hexdigest(-10)
    

    @httpsgithubcomxcainiao httpsgithubcomxcainiao mannequin added the type-security A security issue label Oct 7, 2018
    @ned-deily
    Copy link
    Member

    See also bpo-33729. We need this addressed for 3.6.7.

    @tirkarthi
    Copy link
    Member

    Thanks for the report. Interesting, this is not reproducible on master and latest 3.7 branches though both have different errors but reproducible in latest 3.6 and v3.7.0 . As Ned noted this seems to have been fixed with bpo-33729 but still there is no decision on reverting/keeping the commits made with the linked issue.

    # master

    ./python.exe
    Python 3.8.0a0 (heads/master:7dfbd49671, Oct  7 2018, 16:00:31)
    [Clang 7.0.2 (clang-700.1.81)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import hashlib
    >>> hashlib.shake_128().hexdigest(-10)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: value must be positive

    # upstream/3.7

    ./python.exe
    Python 3.7.1rc1+ (remotes/upstream/3.7:3b699932e5, Oct  7 2018, 21:44:03)
    [Clang 7.0.2 (clang-700.1.81)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import hashlib
    >>> hashlib.shake_128().hexdigest(-10)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    OverflowError: can't convert negative value to unsigned int

    # 3.7.0 segfaults

    ./python.exe
    Python 3.7.0 (tags/v3.7.0:1bf9cc5093, Oct  7 2018, 21:51:43)
    [Clang 7.0.2 (clang-700.1.81)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import hashlib
    >>> hashlib.shake_128().hexdigest(-10)
    [1]    67585 bus error  ./python.exe

    # upstream/3.6 segfaults

    ./python.exe
    Python 3.6.7rc1+ (remotes/upstream/3.6:177254c96f, Oct  7 2018, 21:42:19)
    [GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import hashlib
    >>> hashlib.shake_128().hexdigest(-10)
    [1]    49096 bus error  ./python.exe

    Thanks

    @tirkarthi
    Copy link
    Member

    Sorry Ned, my comment seems to have changed the priority while submitting the comment. I would also propose adding the attached report as a unit test.

    @ned-deily
    Copy link
    Member

    No problem; that's something to watch out for when you get an update conflict message from the bug tracker! Regarding this issue, I believe Serhiy is going to do a PR but perhaps you can work with him on providing the test case.

    @serhiy-storchaka serhiy-storchaka added extension-modules C modules in the Modules dir 3.7 (EOL) end of life 3.8 (EOL) end of life labels Oct 7, 2018
    @serhiy-storchaka serhiy-storchaka self-assigned this Oct 7, 2018
    @serhiy-storchaka serhiy-storchaka added type-crash A hard crash of the interpreter, possibly with a core dump and removed type-security A security issue labels Oct 7, 2018
    @serhiy-storchaka
    Copy link
    Member

    The original crash is nor reproducible in 3.7 and master, but Victor found other example that causes a crash in 3.7 and master.

        import hashlib; hashlib.shake_128().hexdigest(2*64-10)

    Use 2*32-10 on 32-bit platforms.

    I suppose that passing 2**29 on 32-bit platforms will cause problems too. And this is just 512 MiB.

    So this issue affects 3.6, 3.7 and master.

    @httpsgithubcomxcainiao
    Copy link
    Mannequin Author

    httpsgithubcomxcainiao mannequin commented Oct 8, 2018

    I send this to security@python.org.
    Victor Stinner response me.
    "import hashlib; hashlib.shake_128().hexdigest((-1)&2**64-1)" can crash python3.7 and master

    fan@fan:~/github/new$ ./py3.7/bin/python3
    Python 3.7.1rc1+ (heads/3.7:c59e75c, Oct  8 2018, 08:53:13) 
    [GCC 5.4.0 20160609] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import hashlib; hashlib.shake_128().hexdigest((-1)&2**64-1)
    ASAN:SIGSEGV
    =================================================================
    ==29245==ERROR: AddressSanitizer: SEGV on unknown address 0x7f3a50713000 (pc 0x7f3a537994c1 bp 0x7ffd978e27f0 sp 0x7ffd978e1f78 T0)
        #0 0x7f3a537994c0  (/lib/x86_64-linux-gnu/libc.so.6+0x1564c0)
        #1 0x7f3a543df5d0 in __asan_memcpy (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x8c5d0)
        #2 0x7f3a4f5a8603 in memcpy /usr/include/x86_64-linux-gnu/bits/string3.h:53
        #3 0x7f3a4f5a8603 in _PySHA3_KeccakP1600_ExtractLanes /home/fan/github/new/cpython3.7/Modules/_sha3/kcp/KeccakP-1600-opt64.c:342
        #4 0x7f3a4f5a877b in _PySHA3_KeccakP1600_ExtractBytes /home/fan/github/new/cpython3.7/Modules/_sha3/kcp/KeccakP-1600-opt64.c:375
        #5 0x7f3a4f5a8965 in _PySHA3_KeccakWidth1600_SpongeSqueeze /home/fan/github/new/cpython3.7/Modules/_sha3/kcp/KeccakSponge.inc:287
        #6 0x7f3a4f5a92a2 in _SHAKE_digest /home/fan/github/new/cpython3.7/Modules/_sha3/sha3module.c:615
        #7 0x465348 in _PyMethodDef_RawFastCallKeywords Objects/call.c:644
        #8 0x74c83c in _PyMethodDescr_FastCallKeywords Objects/descrobject.c:288
        #9 0x441c3b in call_function Python/ceval.c:4579
        #10 0x441c3b in _PyEval_EvalFrameDefault Python/ceval.c:3110
        #11 0x5a3b1f in _PyEval_EvalCodeWithName Python/ceval.c:3930
        #12 0x5a40c2 in PyEval_EvalCodeEx Python/ceval.c:3959
        #13 0x5a40c2 in PyEval_EvalCode Python/ceval.c:524
        #14 0x605047 in run_mod Python/pythonrun.c:1035
        #15 0x6097c4 in PyRun_InteractiveOneObjectEx Python/pythonrun.c:256
        #16 0x609d65 in PyRun_InteractiveLoopFlags Python/pythonrun.c:120
        #17 0x60ad2b in PyRun_AnyFileExFlags Python/pythonrun.c:78
        #18 0x44d7c5 in pymain_run_file Modules/main.c:427
        #19 0x44d7c5 in pymain_run_filename Modules/main.c:1537
        #20 0x44d7c5 in pymain_run_python Modules/main.c:2626
        #21 0x44d7c5 in pymain_main Modules/main.c:2787
        #22 0x44e33b in _Py_UnixMain Modules/main.c:2822
        #23 0x7f3a5366382f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
        #24 0x442db8 in _start (/home/fan/github/new/py3.7/bin/python3.7+0x442db8)
    
    AddressSanitizer can not provide additional info.
    SUMMARY: AddressSanitizer: SEGV ??:0 ??
    ==29245==ABORTING
    
    (venv) fan@fan:~/github/new$ python
    Python 3.8.0a0 (heads/master:f6c8007, Sep 25 2018, 12:42:29) 
    [GCC 5.4.0 20160609] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import hashlib; hashlib.shake_128().hexdigest((-1)&2**64-1)
    ASAN:SIGSEGV
    =================================================================
    ==29347==ERROR: AddressSanitizer: SEGV on unknown address 0x7f6df36db000 (pc 0x7f6df1a0a210 bp 0x7ffdc8f57a80 sp 0x7ffdc8f57208 T0)
        #0 0x7f6df1a0a20f  (/lib/x86_64-linux-gnu/libc.so.6+0x15720f)
        #1 0x7f6df264f5d0 in __asan_memcpy (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x8c5d0)
        #2 0x7f6ded528643 in memcpy /usr/include/x86_64-linux-gnu/bits/string3.h:53
        #3 0x7f6ded528643 in _PySHA3_KeccakP1600_ExtractLanes /home/fan/github/new/cpython_a/Modules/_sha3/kcp/KeccakP-1600-opt64.c:342
        #4 0x7f6ded5287bb in _PySHA3_KeccakP1600_ExtractBytes /home/fan/github/new/cpython_a/Modules/_sha3/kcp/KeccakP-1600-opt64.c:375
        #5 0x7f6ded5289a5 in _PySHA3_KeccakWidth1600_SpongeSqueeze /home/fan/github/new/cpython_a/Modules/_sha3/kcp/KeccakSponge.inc:287
        #6 0x7f6ded529312 in _SHAKE_digest /home/fan/github/new/cpython_a/Modules/_sha3/sha3module.c:609
        #7 0x7f6ded529312 in _sha3_shake_128_hexdigest_impl /home/fan/github/new/cpython_a/Modules/_sha3/sha3module.c:658
        #8 0x7f6ded529312 in _sha3_shake_128_hexdigest /home/fan/github/new/cpython_a/Modules/_sha3/clinic/sha3module.c.h:116
        #9 0x46b389 in _PyMethodDef_RawFastCallKeywords Objects/call.c:644
        #10 0x81403c in _PyMethodDescr_FastCallKeywords Objects/descrobject.c:288
        #11 0x4416b1 in call_function Python/ceval.c:4600
        #12 0x4416b1 in _PyEval_EvalFrameDefault Python/ceval.c:3186
        #13 0x5ecfbb in PyEval_EvalFrameEx Python/ceval.c:536
        #14 0x5ecfbb in _PyEval_EvalCodeWithName Python/ceval.c:3951
        #15 0x5ed4d2 in PyEval_EvalCodeEx Python/ceval.c:3980
        #16 0x5ed4d2 in PyEval_EvalCode Python/ceval.c:513
        #17 0x68addd in run_mod Python/pythonrun.c:1031
        #18 0x68addd in PyRun_InteractiveOneObjectEx Python/pythonrun.c:256
        #19 0x68b3f5 in PyRun_InteractiveLoopFlags Python/pythonrun.c:120
        #20 0x68b71b in PyRun_AnyFileExFlags Python/pythonrun.c:78
        #21 0x44db6b in pymain_run_stdin Modules/main.c:1182
        #22 0x44db6b in pymain_run_python Modules/main.c:1610
        #23 0x44db6b in pymain_main Modules/main.c:1755
        #24 0x44e39b in _Py_UnixMain Modules/main.c:1792
        #25 0x7f6df18d382f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
        #26 0x446228 in _start (/home/fan/github/new/py/bin/python3.8+0x446228)
    
    AddressSanitizer can not provide additional info.
    SUMMARY: AddressSanitizer: SEGV ??:0 ??
    ==29347==ABORTING
    

    @ned-deily
    Copy link
    Member

    We've reached the cutoff point for 3.7.1rc2 and 3.6.7rc2 and I don't see a PR or a resolution of this for either branch yet. If there's a chance for merged PRs in the next couple of hours, I'll wait a bit longer but otherwise these fixes will have to wait.

    @serhiy-storchaka
    Copy link
    Member

    New changeset 9b8c2e7 by Serhiy Storchaka in branch 'master':
    bpo-34922: Fix integer overflow in the digest() and hexdigest() methods (GH-9751)
    9b8c2e7

    @serhiy-storchaka
    Copy link
    Member

    New changeset 8b040e5 by Serhiy Storchaka in branch '3.7':
    [3.7] bpo-34922: Fix integer overflow in the digest() and hexdigest() methods (GH-9751) (GH-9798)
    8b040e5

    @serhiy-storchaka
    Copy link
    Member

    New changeset 69e6ad6 by Serhiy Storchaka (Miss Islington (bot)) in branch '3.6':
    [3.6] bpo-34922: Fix integer overflow in the digest() and hexdigest() methods (GH-9751) (GH-9798) (GH-9801)
    69e6ad6

    @terryjreedy
    Copy link
    Member

    Should this be closed as fixed?

    @serhiy-storchaka
    Copy link
    Member

    Since it is tagged as a release blocker, I think that only Ned can close it.

    Personally I don't think that this issue is a security issue. digest() and hexdigest() argument usually is a constant. It is unlikely that the crash can be triggered by user data.

    @httpsgithubcomxcainiao
    Copy link
    Mannequin Author

    httpsgithubcomxcainiao mannequin commented Oct 13, 2018

    oh brother, maybe this worth open a cve.

    @ned-deily
    Copy link
    Member

    Serhiy's fixes (thanks!) are now released in 3.7.0rc2 and 3.6.7rc2 so I'm removing the "release blocker" status. If there is nothing more to be done for this issue, can we close it?

    shuoz:

    oh brother, maybe this worth open a cve.

    Note that Serhiy believes that this is not a security issue since it is unlikely that the crash can be triggered by user data. Anyone can cause segfaults or do damage if they have unrestricted access to a Python interpreter; that's a threat model for any language that allows sometime like Python's os.system or subprocess. A better question is can a user of an application written in Python likely cause a DOS or create a privilege escalation. Is that the case here?

    @ned-deily
    Copy link
    Member

    Since there has been no further discussion on this since the fixes were pushed over a year ago, I am declaring this issue resolved. Thanks for everyone's help!

    @ned-deily ned-deily removed their assignment Jan 14, 2020
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life 3.8 (EOL) end of life extension-modules C modules in the Modules dir type-crash A hard crash of the interpreter, possibly with a core dump
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants