archive/zip: not possible to specify deflate compression level

[gopherbot]

by peter.waller:

Discussion link:

https://groups.google.com/forum/#!topic/golang-dev/WtYAcfgPNBw/discussion

There is a TODO to support specifying the compression level.

https://code.google.com/p/go/source/browse/src/pkg/archive/zip/writer.go#17

I do not currently see how it is possible to fit this in to the API without breaking
backwards compatibility, especially since the Compressor interface has no room for
additional parameters.

I would tack a crack at this myself otherwise.

[minux]

Comment 1:

Labels changed: added release-none, repo-main.

Status changed to Accepted.

[bradfitz]

Comment 2:

We could add a struct field to FileHeader, like:
   // CompressionLevel optionally specifies the flate compression level
   // when used with CreateHeader. It isn't populated when reading zip
   // files.
   CompressionLevel int

Owner changed to @adg.

[gopherbot]

Comment 3 by peter.waller:

How does one then get it into the compressor?
https://code.google.com/p/go/source/browse/src/pkg/archive/zip/writer.go#183

[bradfitz]

Comment 4:

Probably special-case the Deflate compressor type first and do our thing for that type.
We could also extend the definition of Compressor to say that compressor can type-assert
an optional interface on top of the io.Writer (like zip.RequestedCompressionLeveler) but
that's gross.  But that could be how we special case it internally with a private
interface.
The biggest question is how to handle the sync.Pool when the flate level can change.
One answer is to not use a sync.Pool for all flate.Writers which aren't level 5. That
will lead to bad performance when zipping lots of items (which is why we added the pool
here in the first place). Another item is having a sync.Pool per flate size. That seems
controversial. I recommend you get an answer from Russ or Rob on that first before you
start doing any work on this.

[dsnet]

Just a thought, since I ran into something related to this.

Perhaps the solution would be simply to allow an existing Compressor to be replaced. Currently, RegisterCompressor panics if the method was already registered. Maybe the library can allow them to be replaced.

This way, the user has more flexibility than simply setting the compression level. They can do:

zip.RegisterCompressor(zip.Deflate, func(w io.Writer) (io.WriteCloser, error) {
    return flate.NewWriter(w, flate.BestSpeed)
})

This gives users more power. For example, I can swap in custom version of the flate library for various purposes (like one that supports parallel compression or has a faster version of flate). One down-side is that this will lose the benefit of the sync.Pool that the library places used writers back in, but it shouldn't be too difficult for a user to re-implement the sync.Pool features.

@bradfitz In archive/zip/register.go, what was the purpose of guarding each Write and Close call of pooledFlateWriter with a mutex?

[gopherbot]

CL https://golang.org/cl/16669 mentions this issue.