Skip to content

Commit

Permalink
dockerTools: Fix layer redundancy.
Browse files Browse the repository at this point in the history
When building an image with multiple layers, files
already included in an underlying layer are supposed to
be excluded from the current layer. However, some subtleties
in the way filepaths are compared seem to be blocking this.

Specifically:
* tar generates relative filepaths with directories ending in '/'
* find generates absolute filepaths with no trailing slashes on directories

That is, paths extracted from the underlying tarball look like:
    nix/store/.../foobar/
whereas the layer being generated uses paths like:
    /nix/store/.../foobar

This patch modifies the output of "tar -t" to match the latter format.
  • Loading branch information
tboettch committed Sep 23, 2016
1 parent 124f25b commit 1e8b69c
Showing 1 changed file with 16 additions and 5 deletions.
21 changes: 16 additions & 5 deletions pkgs/build-support/docker/default.nix
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{ stdenv, lib, callPackage, runCommand, writeReferencesToFile, writeText, vmTools, writeScript
, docker, shadow, utillinux, coreutils, jshon, e2fsprogs, go, pigz }:
, docker, shadow, utillinux, coreutils, jshon, e2fsprogs, go, pigz, findutils }:

# WARNING: this API is unstable and may be subject to backwards-incompatible changes in the future.

Expand Down Expand Up @@ -249,7 +249,7 @@ EOF
then mkPureLayer { inherit baseJson contents extraCommands; }
else mkRootLayer { inherit baseJson fromImage fromImageName fromImageTag contents runAsRoot diskSize extraCommands; });
result = runCommand "${baseName}.tar.gz" {
buildInputs = [ jshon pigz ];
buildInputs = [ jshon pigz coreutils findutils ];

imageName = name;
imageTag = tag;
Expand All @@ -261,6 +261,18 @@ EOF
buildArgs = args;
};
} ''
# Print tar contents:
# 1: Interpreted as relative to the root directory
# 2: With no trailing slashes on directories
# This is useful for ensuring that the output matches the values generated by the "find" command
ls_tar() {
for f in $(tar -tf $1 | xargs realpath -ms --relative-to=.); do
if [ "$f" != "." ]; then
echo "/$f"
fi
done
}
mkdir image
touch baseFiles
if [ -n "$fromImage" ]; then
Expand All @@ -276,7 +288,7 @@ EOF
parentID=$(jshon -e $fromImageName -e $fromImageTag -u < image/repositories)
for l in image/*/layer.tar; do
tar -tf $l >> baseFiles
ls_tar $l >> baseFiles
done
fi
Expand All @@ -297,8 +309,7 @@ EOF
fi
echo Adding layer
tar -tf temp/layer.tar >> baseFiles
sed 's/^\.//' -i baseFiles
ls_tar temp/layer.tar >> baseFiles
comm <(sort -u baseFiles) <(sort -u layerFiles) -1 -3 > newFiles
tar -rpf temp/layer.tar --mtime=0 --no-recursion --files-from newFiles 2>/dev/null || true
Expand Down

0 comments on commit 1e8b69c

Please sign in to comment.