Embedded zipfile fields dependent on absolute position

[spoo]

BPO	26293
Nosy	@Yhg1s, @serhiy-storchaka
PRs	Revert bpo-26293 for zipfile breakage. See also bpo-29094. #1484 [3.6] Revert bpo-26293 for zipfile breakage. See also bpo-29094. (GH-1484). #1485 [3.5] Revert bpo-26293 for zipfile breakage. See also bpo-29094. (GH-1484). #1486
Files	zipfile_concatenated.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2016-10-07.20:15:03.774>
created_at = <Date 2016-02-05.08:52:35.506>
labels = ['3.7', 'type-bug', 'library']
title = 'Embedded zipfile fields dependent on absolute position'
updated_at = <Date 2017-05-06.12:11:22.800>
user = 'https://bugs.python.org/spoo'

bugs.python.org fields:

activity = <Date 2017-05-06.12:11:22.800>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2016-10-07.20:15:03.774>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2016-02-05.08:52:35.506>
creator = 'spoo'
dependencies = []
files = ['42841']
hgrepos = []
issue_num = 26293
keywords = ['patch']
message_count = 14.0
messages = ['259642', '259644', '265484', '267095', '267107', '267598', '278079', '278262', '278264', '278265', '292912', '293155', '293157', '293159']
nosy_count = 5.0
nosy_names = ['twouters', 'alanmcintyre', 'python-dev', 'serhiy.storchaka', 'spoo']
pr_nums = ['1484', '1485', '1486']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue26293'
versions = ['Python 2.7', 'Python 3.5', 'Python 3.6', 'Python 3.7']

[spoo]

Example:

from zipfile import ZipFile
with open('a.zipp', 'wb') as base:
    base.write(b'old\n')
    with ZipFile(base, 'a') as myzip:
        myzip.write('eggs.txt')

If the embedded zip portion of the file is extracted (first four bytes deleted), some fields will be incorrect in the resultant file - commenting out line 3 produces a file that can serve as a comparison. These differences cause issues opening with some zip library implementations.

My best guess is that this is related to this line: https://github.com/python/cpython/blob/master/Lib/zipfile.py#L1459

[spoo]

I'm not familiar with iOS development, but I'd hazard a guess that "some zip library implementations" means a (the official?) iOS zip library.

[serhiy-storchaka]

Yes, this may be a bug. After appending all offsets are absolute file positions. Here is a patch that presumably fixes this issue. Unfortunately I can't write tests for this issue. ZipFile is too lenient and all written tests are passes with unpatched code. Please test the patch manually.

[serhiy-storchaka]

spoo, can you confirm that the patch fixes your issue?

[spoo]

On 06/03/2016 09:27 PM, Serhiy Storchaka wrote:

Serhiy Storchaka added the comment:

spoo, can you confirm that the patch fixes your issue?

----------

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue26293\>

---

Yes, sorry, and thank you for looking into this! It may take me a few
days to be able to test it.

[spoo]

Thank you for looking into this! I don't have my tablet to test with at
the moment, but testing locally with zipinfo there used to be a warning
about missing bytes and now there is none. I will test with the tablet
in a week or two when I get it back.

[spoo]

I confirmed this fixes the issue loading zips on an iPad.

[python-dev]

New changeset 005704f5634f by Serhiy Storchaka in branch '3.5':
Issue bpo-26293: Fixed writing ZIP files that starts not from the start of the
https://hg.python.org/cpython/rev/005704f5634f

New changeset b06e75ae1981 by Serhiy Storchaka in branch '3.6':
Issue bpo-26293: Fixed writing ZIP files that starts not from the start of the
https://hg.python.org/cpython/rev/b06e75ae1981

New changeset 64a19fe3a16a by Serhiy Storchaka in branch 'default':
Issue bpo-26293: Fixed writing ZIP files that starts not from the start of the
https://hg.python.org/cpython/rev/64a19fe3a16a

[python-dev]

New changeset 9a99a88301ef by Serhiy Storchaka in branch '2.7':
Issue bpo-26293: Fixed writing ZIP files that starts not from the start of the
https://hg.python.org/cpython/rev/9a99a88301ef

[serhiy-storchaka]

Thank you for your report and testing.

[serhiy-storchaka]

Unfortunately we should revert this change. It breaks other cases. See bpo-29094.

[serhiy-storchaka]

New changeset 3763ea8 by Serhiy Storchaka in branch 'master':
Revert bpo-26293 for zipfile breakage. See also bpo-29094. (bpo-1484)
3763ea8

[serhiy-storchaka]

New changeset 70dc6a7 by Serhiy Storchaka in branch '3.6':
[3.6] Revert bpo-26293 for zipfile breakage. See also bpo-29094. (GH-1484). (bpo-1485)
70dc6a7

[serhiy-storchaka]

New changeset c8faabc by Serhiy Storchaka in branch '3.5':
[3.5] Revert bpo-26293 for zipfile breakage. See also bpo-29094. (GH-1484). (bpo-1486)
c8faabc