Zipfile breaks if signalled during write()

[and800]

BPO	36434
Nosy	@Yhg1s, @serhiy-storchaka, @and800
PRs	bpo-36434: Properly handle writing errors in ZIP files. #12559 [3.7] bpo-36434: Properly handle writing errors in ZIP files. (GH-12559) #12628

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-03-30.13:54:25.732>
created_at = <Date 2019-03-26.08:59:02.319>
labels = ['3.7', '3.8', 'type-bug', 'library']
title = 'Zipfile breaks if signalled during write()'
updated_at = <Date 2019-03-30.13:54:25.732>
user = 'https://github.com/and800'

bugs.python.org fields:

activity = <Date 2019-03-30.13:54:25.732>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2019-03-30.13:54:25.732>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2019-03-26.08:59:02.319>
creator = 'and800'
dependencies = []
files = []
hgrepos = []
issue_num = 36434
keywords = ['patch']
message_count = 4.0
messages = ['338863', '339178', '339198', '339200']
nosy_count = 4.0
nosy_names = ['twouters', 'alanmcintyre', 'serhiy.storchaka', 'and800']
pr_nums = ['12559', '12628']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue36434'
versions = ['Python 3.7', 'Python 3.8']

[and800]

Consider a simple write to a zip file:

import zipfile

with zipfile.ZipFile('/workdir/archive.zip', 'w', compression=zipfile.ZIP_DEFLATED) as zip_archive:
zip_archive.write('/workdir/data.csv', arcname='data.csv')
print('exiting from context manager...')

If a signal handler is fired and raises an exception during certain points of write() execution, such an error occurs (py 3.7.2):

Traceback (most recent call last):
  File "zipissue.py", line 4, in <module>
    zip_archive.write('/workdir/data.csv', arcname='data.csv')
  File "/usr/local/lib/python3.7/zipfile.py", line 1744, in write
    shutil.copyfileobj(src, dest, 1024*8)
  File "/usr/local/lib/python3.7/zipfile.py", line 1107, in close
    buf = self._compressor.flush()
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "zipissue.py", line 5, in <module>
    print('exiting from context manager...')
  File "/usr/local/lib/python3.7/zipfile.py", line 1265, in __exit__
    self.close()
  File "/usr/local/lib/python3.7/zipfile.py", line 1798, in close
    raise ValueError("Can't close the ZIP file while there is "
ValueError: Can't close the ZIP file while there is an open writing handle on it. Close the writing handle before closing the zip.
Exception ignored in: <function ZipFile.__del__ at 0x7fbeea4fcd08>
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/zipfile.py", line 1789, in __del__
  File "/usr/local/lib/python3.7/zipfile.py", line 1798, in close
ValueError: Can't close the ZIP file while there is an open writing handle on it. Close the writing handle before closing the zip.

Before any write the ZipFile._writing flag is set, and that flag is cleared at _ZipWriteFile.close(). But if signalled inside _ZipWriteFile.close() we are moving to a broken state: we don't write anything anymore, but ZipFile._writing is still set. Therefore we cannot clearly close ZipFile. As ZipFile contextmanager swallows KeyboardInterrupt and produces an exception of Exception type, this leads to the impossibility of proper program shutdown.

I believe that by simply moving ZipFile._writing = False in _ZipWriteFile.close() to some finally block, this issue will be solved safely.

[serhiy-storchaka]

New changeset 2524fde by Serhiy Storchaka in branch 'master':
bpo-36434: Properly handle writing errors in ZIP files. (GH-12559)
2524fde

[serhiy-storchaka]

New changeset 4724ba9 by Serhiy Storchaka (Miss Islington (bot)) in branch '3.7':
bpo-36434: Properly handle writing errors in ZIP files. (GH-12559) (GH-12628)
4724ba9

[serhiy-storchaka]

Thank you for your report Andriy. 3.5 and 3.6 take only security bug fixes.