tarfile creates output that appears to omit files

[mcr]

BPO	40914
Nosy	@zware, @mcr

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-06-15.19:11:24.717>
created_at = <Date 2020-06-08.21:15:00.298>
labels = ['3.7', 'invalid', 'type-bug', 'library']
title = 'tarfile creates output that appears to omit files'
updated_at = <Date 2020-06-15.19:11:24.716>
user = 'https://github.com/mcr'

bugs.python.org fields:

activity = <Date 2020-06-15.19:11:24.716>
actor = 'zach.ware'
assignee = 'none'
closed = True
closed_date = <Date 2020-06-15.19:11:24.717>
closer = 'zach.ware'
components = ['Library (Lib)']
creation = <Date 2020-06-08.21:15:00.298>
creator = 'mcr314'
dependencies = []
files = []
hgrepos = []
issue_num = 40914
keywords = []
message_count = 2.0
messages = ['371045', '371047']
nosy_count = 2.0
nosy_names = ['zach.ware', 'mcr314']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue40914'
versions = ['Python 3.6', 'Python 3.7']

[mcr]

The simplest tarcopy program seems to result in output that GNU tar, bsdtar, and even Emacs tar-mode is unable to correctly process.
It appears that the resulting tar file is missing files, but examination of the raw output shows they might be there, but just corrupt.
GNU tar actually complains while reading the file.
https://github.com/mcr/python3-tar-copy-failure

has a test case. Here is the stupid code to reproduce it:

import tarfile
out = tarfile.open(name="./t2.tar", mode="w", format=tarfile.PAX_FORMAT)
with tarfile.open("./t1.tar") as tar:
    for file in tar.getmembers():
        print (file.name)
        out.addfile(file)
out.close()

This has been confirmed on python 3.6.9 (Ubuntu 18.04 LTS), and python 3.7.3 (Devuan Beowulf). It seems to omit different files on 32-bit and 64-bit systems.

[zware]

Note that TarFile.getmembers() is documented to return TarInfo objects, which are documented as explicitly not including file data. Try replacing out.addfile(file) with out.addfile(file, tar.extractfile(file)).