Improper gzip compression if output file extension is not "gz"

[Delgan]

BPO	32698
Nosy	@vadmium, @Delgan
Files	test.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-01-29.19:33:55.006>
created_at = <Date 2018-01-28.18:37:33.292>
labels = ['invalid', 'type-bug', 'library']
title = 'Improper gzip compression if output file extension is not "gz"'
updated_at = <Date 2018-01-29.19:33:55.005>
user = 'https://github.com/Delgan'

bugs.python.org fields:

activity = <Date 2018-01-29.19:33:55.005>
actor = 'Delgan'
assignee = 'none'
closed = True
closed_date = <Date 2018-01-29.19:33:55.006>
closer = 'Delgan'
components = ['Library (Lib)']
creation = <Date 2018-01-28.18:37:33.292>
creator = 'Delgan'
dependencies = []
files = ['47414']
hgrepos = []
issue_num = 32698
keywords = []
message_count = 3.0
messages = ['310978', '311018', '311160']
nosy_count = 2.0
nosy_names = ['martin.panter', 'Delgan']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue32698'
versions = ['Python 3.6']

[Delgan]

Hello.

The following code produces a improper compressed "test.txt.gzip" file:

    import gzip
    import shutil
    
    input_path = "test.txt"
    output_path = input_path + ".gzip"
    
    with open(input_path, 'w') as file:
        file.write("abc" * 10)
    
    with gzip.open(output_path, 'wb') as f_out:
        with open(input_path, 'rb') as f_in:
            shutil.copyfileobj(f_in, f_out)

Although the content can be read correctly using gzip.open(outputh_path, 'rb'), it cannot be correctly opened using software like 7-Zip or WinRar.
If I open the "test.txt.gzip" file, it contains another "test.txt.gzip" file. If I change the code to use ".gz" extension and then open "test.txt.gz", it contains the expected "test.txt" file.
The contained "test.txt.gzip" is actually the same (at bytes level) that "test.txt", just the filename differs which causes tools like 7-Zip to mess up.

The bug is not present using compressions functions from "bz2" and "lzma" modules. I can use custom extension, it still can be (un)compressed without issue.

As to why I need to use an extension differents from ".gz": I would like to compress arbitrary ".tar" file given in input to ".tgz". I wish the user could open the file in his favorite software archiver and see that it contains a ".tar" file, rather than he does not understand why it contains the same ".tgz" file.

[vadmium]

According to the documentation, you can use the lower-level GzipFile constructor’s “filename” argument:

>>> with open(output_path, 'wb') as f_out, \
...     gzip.GzipFile(fileobj=f_out, mode='wb', filename=input_path) as f_out, \
...     open(input_path, 'rb') as f_in:
...         shutil.copyfileobj(f_in, f_out)
... 
>>> import os
>>> os.system("7z l test.txt.gzip")
[. . .]
   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  

---

2018-01-28 22:23:16 ..... 30 34 test.txt
------------------- ----- ------------ ------------ ------------------------

[Delgan]

Thanks @martin.panter for your response.

I will close this issue as "not a bug" as there is a workaround and as the current behavior could be deduced by reading carefully the entire documentation.