zlib.error with tarfile.open

[jvoisin]

BPO	39039
Nosy	@tiran, @ethanfurman, @ambv, @jdevries3133
PRs	bpo-39039: tarfile raises descriptive exception from zlib.error #27766 [3.10] bpo-39039: tarfile raises descriptive exception from zlib.error (GH-27766) #28613 [3.9] bpo-39039: tarfile raises descriptive exception from zlib.error (GH-27766) #28614
Files	crash-c10c9839d987fa0df6912cb4084f43f3ce08ca82

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-09-29.10:58:55.030>
created_at = <Date 2019-12-13.16:00:48.662>
labels = ['type-bug', 'library', '3.9', '3.10', '3.11']
title = 'zlib.error with tarfile.open'
updated_at = <Date 2021-09-29.10:58:55.028>
user = 'https://bugs.python.org/jvoisin'

bugs.python.org fields:

activity = <Date 2021-09-29.10:58:55.028>
actor = 'lukasz.langa'
assignee = 'none'
closed = True
closed_date = <Date 2021-09-29.10:58:55.030>
closer = 'lukasz.langa'
components = ['Library (Lib)']
creation = <Date 2019-12-13.16:00:48.662>
creator = 'jvoisin'
dependencies = []
files = ['48774']
hgrepos = []
issue_num = 39039
keywords = ['patch']
message_count = 9.0
messages = ['358337', '358340', '358341', '399811', '399989', '402834', '402845', '402847', '402848']
nosy_count = 5.0
nosy_names = ['christian.heimes', 'ethan.furman', 'lukasz.langa', 'jvoisin', 'jack__d']
pr_nums = ['27766', '28613', '28614']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue39039'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

[jvoisin]

The attached file produces the following stacktrace when opened via tarfile.open, on Python 3.7.5rc1:

$ cat test.py 
import sys
import tarfile

tarfile.open(sys.argv[1])
$ python3 test.py ./crash-c10c9839d987fa0df6912cb4084f43f3ce08ca82
Traceback (most recent call last):
  File "test.py", line 4, in <module>
    tarfile.open(sys.argv[1])
  File "/usr/lib/python3.7/tarfile.py", line 1573, in open
    return func(name, "r", fileobj, **kwargs)
  File "/usr/lib/python3.7/tarfile.py", line 1645, in gzopen
    t = cls.taropen(name, mode, fileobj, **kwargs)
  File "/usr/lib/python3.7/tarfile.py", line 1621, in taropen
    return cls(name, mode, fileobj, **kwargs)
  File "/usr/lib/python3.7/tarfile.py", line 1484, in __init__
    self.firstmember = self.next()
  File "/usr/lib/python3.7/tarfile.py", line 2289, in next
    tarinfo = self.tarinfo.fromtarfile(self)
  File "/usr/lib/python3.7/tarfile.py", line 1094, in fromtarfile
    buf = tarfile.fileobj.read(BLOCKSIZE)
  File "/usr/lib/python3.7/gzip.py", line 276, in read
    return self._buffer.read(size)
  File "/usr/lib/python3.7/_compression.py", line 68, in readinto
    data = self.read(len(byte_view))
  File "/usr/lib/python3.7/gzip.py", line 471, in read
    uncompress = self._decompressor.decompress(buf, size)
zlib.error: Error -3 while decompressing data: invalid distances se

[tiran]

This file is also an invalid tar file:

$ tar xf crash-c10c9839d987fa0df6912cb4084f43f3ce08ca82 

gzip: stdin: invalid compressed data--format violated
tar: Child returned status 1
tar: Error is not recoverable: exiting now

[jvoisin]

Sure, but as a user, I would expect a better exception, like ValueError or ReadError, along with an error message, instead of an unexpected zlib exception.

[jdevries3133]

@jvoisin I am able to reproduce the problem when I download your script, but I am having a hard time reproducing it by passing corrupt archives to tarfile.open. How exactly was this file corrupted? I am trying to figure out if there are any similar implementation leaks / poor error messages in similar scenarios so I can do my best to patch them all.

You can see the reproduction scripts I am using here to get a better idea of what I have been trying. Be forewarned, they are pretty gnarly!

https://gist.github.com/jdevries3133/acbb5ba2a19093d3bcc214733ef85e5a

[jvoisin]

The file was created with a fuzzer, like the one described in https://dustri.org/b/fuzzing-python-in-python-and-doing-it-fast.html

[ambv]

New changeset b6fe857 by Jack DeVries in branch 'main':
bpo-39039: tarfile raises descriptive exception from zlib.error (GH-27766)
b6fe857

[ambv]

New changeset d6b69f2 by Łukasz Langa in branch '3.10':
[3.10] bpo-39039: tarfile raises descriptive exception from zlib.error (GH-27766) (GH-28613)
d6b69f2

[ambv]

New changeset 7bff4d3 by Łukasz Langa in branch '3.9':
[3.9] bpo-39039: tarfile raises descriptive exception from zlib.error (GH-27766) (GH-28614)
7bff4d3

[ambv]

Thanks for the fix, Jack! ✨ 🍰 ✨

Since the change translated zlib.error to tarfile.ReadError which already has to be handled by user code, it's strictly decreasing the surface of necessary exception handling. So, treating this as a bug fix, I backported this to 3.9 and 3.10 as well.