Builder - Add zip support to ADD

[thecloudtaylor]

Many windows components are available as .zip files (for example .net core, PowerShell etc...) because ADD does not support unpacking of those files code execution in the container is required creating additional unneeded layers.

Linux Example w/tar
FROM alpine
ADD foo.tar.gz /bar/

Result
foo is unpacked under the new bar directory

Windows Example w/tar
FROM microsoft/nanoserver
ADD foo.tar.gz /bar/

Result
foo is unpacked under the new bar directory

Windows Example w/zip
FROM microsoft/nanoserver
ADD foo.zip /bar/

Result
foo.zip is copied to the /bar directory i.e. bar/foo.zip :(

[friism]

I think this is a dupe of #15036

As mentioned in that issue, ADD id not recommend best practice so I'm not sure it's worth extending it.

[thecloudtaylor]

I'm not married to ADD as the keyword but the scenario is pretty valuable.

[friism]

@taylorb-microsoft the problem is that you typically don't want exactly what's in a .zip. Instead, you want the zip, do some transformation on the contents (eg. compile source code), and then get rid of anything from the zip you don't need. To keep image-size minimal, you want to do that in a single layer.

Here's an example: https://github.com/docker-library/golang/blob/master/1.8/windows/windowsservercore/Dockerfile#L15

Details here: https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/#add-or-copy

Do you have a specific use-case in mind?

[unclejack]

ADD has been frozen a few years ago. It's been discussed a bunch of times already on this issue tracker. The issue with ADD is that it has too much magic and it's pretty hard to keep track of all the edge cases when using the instruction in a Dockerfile.

ADD shouldn't be extended at all. Instead, new keywords which state the action to be taken explicitly should be introduced. COPY was the first instruction which was meant to help deprecate ADD in favor of specialized instructions. This means we should split up ADD into separate instructions. COPY already copies files. UNPACK or EXTRACT should handle various archives (.tar.gz, .tar.bz2, .tar.xz, .zip). Another instruction should exist for the retrieval of files from remote sources. This fetch instruction could also have a parameter which allow the archive to be extracted or just retrieved for storage as it is.

[thecloudtaylor]

Thanks, we are looking at bringing better compression tools support into Windows but it does seem like being able to accomplish these operations without starting a container might be beneficial.

[unclejack]

@taylorb-microsoft Perhaps this is something to discuss to come up with a new instruction for the Dockerfile. It really is a bad idea to overload ADD any further with any new functionality.

There's no reason why such functionality couldn't be added using a different new instruction (UNPACK, EXTRACT and so on). It's just that it needs a bit more effort to make sure we take into account all the various scenarios.

[synergiator]

But how do you integrate ZIP files then? AFAIK, the COPY instruction will create an image layer you do not need afterwards in most cases for this scenario.

This is a waste of space which will become noticable for large archives.

https://devops.stackexchange.com/questions/3172/copy-and-zip-files-in-dockerfile

[thaJeztah]

Automatic extraction (with ADD) only works for local archives; if you only need the files, not the archive, it's better to just add the extracted files, instead of a zip, and later extracting during docker build.

If you're adding a remote archive; use a multi stage build, and decompress in an intermediate build-stage

[tandeday]

Automatic extraction (with ADD) only works for local archives; if you only need the files, not the archive, it's better to just add the extracted files, instead of a zip, and later extracting during docker build.

I need to be able to build under Windows and the NTFS filesystem mapping may not preserve the original metadata exactly, hence I would like to unpack an archive instead.