integer overflow in itertools.permutations

[pkt]

BPO	23363
Nosy	@serhiy-storchaka
Files	poc_permutations.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-02-04.06:10:39.528>
created_at = <Date 2015-02-01.13:54:13.258>
labels = ['type-crash']
title = 'integer overflow in itertools.permutations'
updated_at = <Date 2015-02-04.06:10:39.527>
user = 'https://bugs.python.org/pkt'

bugs.python.org fields:

activity = <Date 2015-02-04.06:10:39.527>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2015-02-04.06:10:39.528>
closer = 'serhiy.storchaka'
components = []
creation = <Date 2015-02-01.13:54:13.258>
creator = 'pkt'
dependencies = []
files = ['37962']
hgrepos = []
issue_num = 23363
keywords = []
message_count = 7.0
messages = ['235170', '235221', '235225', '235309', '235369', '235377', '235379']
nosy_count = 4.0
nosy_names = ['Arfrever', 'python-dev', 'serhiy.storchaka', 'pkt']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue23363'
versions = ['Python 2.7', 'Python 3.3', 'Python 3.4', 'Python 3.5']

[pkt]

# Bug
# ---
#
# static PyObject *
# permutations_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
# {
# ...
# 1 cycles = PyMem_Malloc(r * sizeof(Py_ssize_t));
# ...
# for (i=0 ; i<r ; i++)
# 2 cycles[i] = n - i;
#
# 1. if r=2^30, then r*sizeof(Py_ssize_t)=2^30*2^2=0 (modulo 2^32), so malloc
# allocates a 0 byte buffer
# 2. r=2^30>0, so we write well beyond the buffer's end
#
# Crash
# -----
#
# Breakpoint 1, permutations_new (type=0x83394e0 <permutations_type>, args=('A', 1073741824), kwds=0x0) at ./Modules/itertoolsmodule.c:3012
# ...
\bpo-3044 indices = PyMem_Malloc(n * sizeof(Py_ssize_t));
# (gdb) print r
# $2 = 1073741824
# (gdb) print r*4
# $3 = 0
# (gdb) c
# Continuing.
#
# Program received signal SIGSEGV, Segmentation fault.
# 0x08230900 in permutations_new (type=0x83394e0 <permutations_type>, args=('A', 1073741824), kwds=0x0) at ./Modules/itertoolsmodule.c:3054
\bpo-3054 cycles[i] = n - i;
#
# OS info
# -------

# 
# % ./python -V
# Python 3.4.1
#  
# % uname -a
# Linux ubuntu 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 15:31:16 UTC 2013 i686 i686 i386 GNU/Linux
#  
 
import itertools as it
it.permutations("A", 2**30)

[python-dev]

New changeset 7133582b6769 by Benjamin Peterson in branch '3.3':
check for overflows in permutations() and product() (closes bpo-23363, closes bpo-23364)
https://hg.python.org/cpython/rev/7133582b6769

New changeset 9ae055c3db32 by Benjamin Peterson in branch '3.4':
merge 3.3 (bpo-23364, bpo-23363)
https://hg.python.org/cpython/rev/9ae055c3db32

New changeset 31dc5a40d2ab by Benjamin Peterson in branch 'default':
merge 3.4 (bpo-23364, bpo-23363)
https://hg.python.org/cpython/rev/31dc5a40d2ab

New changeset acc2c3479f2e by Benjamin Peterson in branch '2.7':
check for overflows in permutations() and product() (closes bpo-23363, closes bpo-23364)
https://hg.python.org/cpython/rev/acc2c3479f2e

[serhiy-storchaka]

An overflow in n * sizeof(Py_ssize_t) is not possible because n is the length of already allocated array of pointers.

+ with self.assertRaises(OverflowError):
+ permutations("A", 2**30)

The test needs 4GiB. May be use 2**29?

+ with self.assertRaises(OverflowError):
+ permutations("A", 2, 2**30)

permutations() takes at most 2 arguments.

[python-dev]

New changeset 356ed025dbae by Serhiy Storchaka in branch '3.3':
Issues bpo-23363, bpo-23364, bpo-23365, bpo-23366: Fixed itertools overflow tests.
https://hg.python.org/cpython/rev/356ed025dbae

New changeset 98c720c3e061 by Serhiy Storchaka in branch '3.4':
Issues bpo-23363, bpo-23364, bpo-23365, bpo-23366: Fixed itertools overflow tests.
https://hg.python.org/cpython/rev/98c720c3e061

New changeset 4cb316fe6bf2 by Serhiy Storchaka in branch 'default':
Issues bpo-23363, bpo-23364, bpo-23365, bpo-23366: Fixed itertools overflow tests.
https://hg.python.org/cpython/rev/4cb316fe6bf2

[Arfrever]

The last fix should be applied also in 2.7 branch.

[python-dev]

New changeset 887526ebb013 by Serhiy Storchaka in branch '2.7':
Issues bpo-23363, bpo-23364, bpo-23365, bpo-23366: Fixed itertools overflow tests.
https://hg.python.org/cpython/rev/887526ebb013

[serhiy-storchaka]

Thanks Arfrever.