Add --fast, --best to the gzip CLI

[matrixise]

BPO	34969
Nosy	@vstinner, @pmp-p, @serhiy-storchaka, @matrixise, @JulienPalard
PRs	bpo-34969: Add --fast, --best on the gzip CLI #9833

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/matrixise'
closed_at = <Date 2018-11-03.21:18:09.212>
created_at = <Date 2018-10-13.05:45:43.524>
labels = ['3.8']
title = 'Add --fast, --best to the gzip CLI'
updated_at = <Date 2018-11-03.21:18:09.211>
user = 'https://github.com/matrixise'

bugs.python.org fields:

activity = <Date 2018-11-03.21:18:09.211>
actor = 'mdk'
assignee = 'matrixise'
closed = True
closed_date = <Date 2018-11-03.21:18:09.212>
closer = 'mdk'
components = []
creation = <Date 2018-10-13.05:45:43.524>
creator = 'matrixise'
dependencies = []
files = []
hgrepos = []
issue_num = 34969
keywords = ['patch']
message_count = 10.0
messages = ['327626', '327628', '327631', '327637', '328158', '328799', '328801', '328803', '328825', '329201']
nosy_count = 5.0
nosy_names = ['vstinner', 'pmpp', 'serhiy.storchaka', 'matrixise', 'mdk']
pr_nums = ['9833']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue34969'
versions = ['Python 3.8']

[matrixise]

the gzip module has a CLI but this one does not allow to specify the compression method (slow, fast)

[JulienPalard]

Hi Stéphane thanks for the proposal and the PR.

But are those options usefull in real life? (I may be biased as a Linux user).

I see this gzip CLI usefull to decompress a gzip file on platforms not having a gzip program installed, but I don't think it's usefull to compress. (Yet I'm OK with the current status-quo: If we allow to decompress, let's allow to compress, for consistency).

But why drifting from "let's allow to compress just for consistency" to "let's replace the gzip command"?

I mean, except is it's usefull in some cases that I don't see.

Also so does it mean we'll have to add the 16 other gzip options at the end?

[pmp-p]

Hi, on platform without gzip ( there are some , including some widely used OS eg: bazelbuild/rules_docker#507 )
ability to use python gzip cli is extremely usefull as a fallback.

Though as discussed on irc default compression to 6 is a good tradeoff for the basic options fast (-1) / best(-9) and require less memory for decompression on target (could be embedded).

[matrixise]

Julien,

Currently, the default compresslevel for gzip.open, GzipFile is 9, for the best method compression. Maybe we could define 6 as the tradeoff and specify 1 (for fast), 9 (for best), 6 (tradeoff).

Maybe in an other issue?

[serhiy-storchaka]

I'm not sure those options are such useful. If you want to control more details, it is not hard to write a tiny Python script. Python is a programming language on the whole.

But if I add options for controlling the compression level, I would add options -1 ... -9. I never used verbose forms --fast and --best.

[vstinner]

But if I add options for controlling the compression level, I would add options -1 ... -9. I never used verbose forms --fast and --best.

If we add options, I would prefer to only add --fast and --best which are easy to understand. I really have no idea of the difference between -3 and -4 for example. In practice, I don't think that anyone uses these -N options on the common line.

To be honest, I never passed any option to gzip: I always use "gzip file" to get "file.gz". I don't really care of it's file. I just hope that it's smaller :-)

But I'm not against add --best and --fast. By the way, on Linux, gzip default compression level is 6 whereas Python uses 9 by default. I agree to make Python more consistent with Unix tools. In that case, again, it makes sense to add an option to get again --best (level 9).

[serhiy-storchaka]

If we add this options to the gzip CLI, I would suggest to add them to bzip2 and lzma CLI in the same PR. And maybe open separate issues for the zipfile and tarfile CLI,

[vstinner]

man lzma:

   --fast
   --best These are somewhat misleading aliases for  -0  and  -9,  respec‐
          tively.   These  are  provided  only for backwards compatibility
          with LZMA Utils.  Avoid using these options.

man bzip2:

   -1 (or --fast) to -9 (or --best)
          Set  the  block size to 100 k, 200 k ..  900 k when compressing.
          Has no effect when decompressing.  See MEMORY MANAGEMENT  below.
          The --fast and --best aliases are primarily for GNU gzip compat‐
          ibility.  In particular, --fast  doesn't  make  things  signifi‐
          cantly faster.  And --best merely selects the default behaviour.

[matrixise]

Hi @serhiy

I would like to add them on lzma, bz2, zipfile and tarfile.

[JulienPalard]

New changeset 3e28eed by Julien Palard (Stéphane Wirtel) in branch 'master':
bpo-34969: Add --fast, --best on the gzip CLI (GH-9833)
3e28eed