bz2/lzma: Compressor/decompressor crash if __init__ is not called

[izbyshev]

BPO	34729
Nosy	@berkerpeksag, @serhiy-storchaka, @izbyshev
Superseder	bpo-23224: bz2/lzma: Compressor/Decompressor objects are only initialized in init

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-09-18.23:30:54.510>
created_at = <Date 2018-09-18.22:49:05.062>
labels = ['extension-modules', '3.7', '3.8', 'type-crash']
title = 'bz2/lzma: Compressor/decompressor crash if __init__ is not called'
updated_at = <Date 2018-09-18.23:35:38.240>
user = 'https://github.com/izbyshev'

bugs.python.org fields:

activity = <Date 2018-09-18.23:35:38.240>
actor = 'berker.peksag'
assignee = 'none'
closed = True
closed_date = <Date 2018-09-18.23:30:54.510>
closer = 'izbyshev'
components = ['Extension Modules']
creation = <Date 2018-09-18.22:49:05.062>
creator = 'izbyshev'
dependencies = []
files = []
hgrepos = []
issue_num = 34729
keywords = []
message_count = 5.0
messages = ['325691', '325693', '325694', '325695', '325696']
nosy_count = 3.0
nosy_names = ['berker.peksag', 'serhiy.storchaka', 'izbyshev']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '23224'
type = 'crash'
url = 'https://bugs.python.org/issue34729'
versions = ['Python 2.7', 'Python 3.6', 'Python 3.7', 'Python 3.8']

[izbyshev]

The compressor/decompressor classes from bz2 and lzma modules rely on __init__() for initialization, but it is not guaranteed to be called. Method calls on an uninitialized object crash:

>>> from bz2 import BZ2Compressor as C
>>> c = C.__new__(C)
>>> c.compress(b'')
Segmentation fault (core dumped)

I see two ways to fix this:

1. Move some initialization (notably, for "lock" field) to __new__() and add initialization checks to other methods. This should be backwards-compatible.

2. Move all initialization to __new__(). Since compressor/decompressor classes are not subclassable, it'll break only code than repeatedly calls __init__() on the same object. The simplicity of the fix might outweigh the necessity to support such code.
   (However, in 2.7, classes in bz2 *are* subclassable; lzma is not present in 2.7).

Which way is more preferable?

[berkerpeksag]

I think we usually went with option 1 when we fixed similar issues in the past.

See also bpo-23224 for the same problem in *Decompressor classes of lzma and bz2 modules. It looks like the attached PR to that issue went with option 2: PR 7822.

Perhaps we can combine this and bpo-23224.

[izbyshev]

I somehow failed to notice bpo-23224 when I searched the tracker. You're right, it's the same, and, moreover, PR 7822 fixes problem with both compressors and decompressors (though it includes tests only for the latter for some reason).

I think that this report should be closed as duplicate, but should we also change the title of bpo-23224 to be more general?

[izbyshev]

Reclosing (browser cache problem).

[berkerpeksag]

We can definitely make the title of that issue more descriptive. Feel free to change it -- IIRC, you don't need additional permissions to change the title of an issue. Thanks!