crash in methods of a subclass of _collections.deque with a bad __new__()

[orenmn]

BPO	31608
Nosy	@rhettinger, @benjaminp, @serhiy-storchaka, @orenmn, @miss-islington
PRs	bpo-31608: Fix a crash in methods of a subclass of _collections.deque with a bad __new__() #3788 [3.7] closes bpo-31608: Fix a crash in methods of a subclass of _collections.deque with a bad __new__(). (GH-3788) #9177 [2.7] closes bpo-31608: Fix a crash in methods of a subclass of _collections.deque with a bad __new__(). #9179 [3.6] closes bpo-31608: Fix a crash in methods of a subclass of _collections.deque with a bad __new__(). #9178

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/rhettinger'
closed_at = <Date 2018-09-11.18:46:58.886>
created_at = <Date 2017-09-27.10:43:58.369>
labels = ['extension-modules', '3.7', 'type-crash']
title = 'crash in methods of a subclass of _collections.deque with a bad __new__()'
updated_at = <Date 2020-12-08.13:23:48.376>
user = 'https://github.com/orenmn'

bugs.python.org fields:

activity = <Date 2020-12-08.13:23:48.376>
actor = 'benjamin.peterson'
assignee = 'rhettinger'
closed = True
closed_date = <Date 2018-09-11.18:46:58.886>
closer = 'benjamin.peterson'
components = ['Extension Modules']
creation = <Date 2017-09-27.10:43:58.369>
creator = 'Oren Milman'
dependencies = []
files = []
hgrepos = []
issue_num = 31608
keywords = ['patch']
message_count = 7.0
messages = ['303125', '303133', '304000', '325049', '325054', '325055', '325057']
nosy_count = 5.0
nosy_names = ['rhettinger', 'benjamin.peterson', 'serhiy.storchaka', 'Oren Milman', 'miss-islington']
pr_nums = ['3788', '9177', '9179', '9178']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue31608'
versions = ['Python 2.7', 'Python 3.6', 'Python 3.7']

[orenmn]

The following code causes the interpreter to crash:

import _collections
class BadDeque(_collections.deque):
    def __new__(cls, *args):
        if len(args):
            return 42
        return _collections.deque.__new__(cls)

BadDeque() * 42

(The interpreter would crash also if we replaced 'BadDeque() * 42' with
'BadDeque() + _collections.deque([42])'.)

This is because deque_copy() (in Modules/_collectionsmodule.c) returns whatever
BadDeque() returned, without verifying it is a deque.
deque_repeat() assumes that deque_copy() returned a deque, and passes it to
deque_inplace_repeat(), which assumes it is a deque, and crashes.

(Similarly, deque_concat() assumes that deque_copy() returned a deque, which
is the reason for the other crash.)

ISTM it is a very unlikely corner case, so that adding a test (as well as
a NEWS.d item) for it is unnecessary.
What do you think?

[serhiy-storchaka]

All other sequence objects return an instance of the base class rather than a subclass. list, tuple, str, bytes, bytearray, array. Only deque tries to create an instance of a subclass.

[serhiy-storchaka]

I meant that if make deque.copy(), deque.__add__() and deque.__mul__() returning an exact deque for subclasses, this would make the deque class more consistent with other collections and solve this issue. This could even make them (almost) atomic.

[benjaminp]

New changeset 24bd50b by Benjamin Peterson (Oren Milman) in branch 'master':
closes bpo-31608: Fix a crash in methods of a subclass of _collections.deque with a bad __new__(). (GH-3788)
24bd50b

[miss-islington]

New changeset 536e45a by Miss Islington (bot) in branch '3.7':
closes bpo-31608: Fix a crash in methods of a subclass of _collections.deque with a bad __new__(). (GH-3788)
536e45a

[benjaminp]

New changeset ccbbdd0 by Benjamin Peterson in branch '3.6':
[3.6] closes bpo-31608: Fix a crash in methods of a subclass of _collections.deque with a bad __new__(). (GH-9178)
ccbbdd0

[benjaminp]

New changeset 253279c by Benjamin Peterson in branch '2.7':
[2.7] closes bpo-31608: Fix a crash in methods of a subclass of _collections.deque with a bad __new__(). (GH-9179)
253279c