Add more integration tests for deltas

This adds two new integration tests:

* TestDeltaSize: this is meant to catch regressions on delta sizes. It
  generates deltas and compare their sizes with the delta sizes we get
  as of now. If the size increased, the test fails.
* TestDeltaCorrectness: checks if applying a delta indeed results in the
  same image as we had originally.

A number of different test cases (different images with distinct
features) are tested for each of these integration tests.

Signed-off-by: Leandro Motta Barros <leandro@balena.io>
Change-type: patch

integration/image/main_test.go

@@ -6,6 +6,7 @@ import (
 	"testing"
 
 	"github.com/docker/docker/internal/test/environment"
+	"github.com/docker/docker/internal/test/registry"
 )
 
 var testEnv *environment.Execution
@@ -31,3 +32,12 @@ func setupTest(t *testing.T) func() {
 	environment.ProtectAll(t, testEnv)
 	return func() { testEnv.Clean(t) }
 }
+
+// setupTemporaryTestRegistry creates a temporary image registry to be used
+// during testing. Returns a function that must be called to tear down this
+// registry.
+func setupTemporaryTestRegistry(t *testing.T) func() {
+	reg := registry.NewV2(t)
+	reg.WaitReady(t)
+	return reg.Close
+}

integration/testdata/delta/000.Dockerfile

@@ -0,0 +1,3 @@
+FROM scratch
+
+COPY 000.data /

integration/testdata/delta/001.Dockerfile

@@ -0,0 +1,3 @@
+FROM 127.0.0.1:5000/000
+
+COPY 000.data /001/000.data

integration/testdata/delta/002.Dockerfile

@@ -0,0 +1,3 @@
+FROM 127.0.0.1:5000/000
+
+COPY 002.data /

integration/testdata/delta/003.Dockerfile

@@ -0,0 +1,6 @@
+FROM 127.0.0.1:5000/001
+
+COPY 000.data /003/000-a.data
+COPY 001.data /003/000-b.data
+COPY 002.data /001/000.data
+COPY 000.data /003/000-c.data

integration/testdata/delta/004.Dockerfile

@@ -0,0 +1,10 @@
+FROM scratch
+
+COPY 001.data /
+COPY 002.data 003.data 004.data /
+COPY 005.data 006.data /
+COPY 007.data /
+COPY 008.data /
+COPY 009.data 010.data /
+COPY 011.data /
+COPY 012.data 013.data 014.data 015.data /

integration/testdata/delta/005.Dockerfile

@@ -0,0 +1,10 @@
+FROM scratch
+
+COPY 001.data /
+COPY 002.data 003.data 004.data /
+COPY 005.data 006.data /
+COPY 007.data /
+COPY 008.data /
+COPY 009.data 010.data /
+COPY 011.data /
+COPY 012.data 013.data 014.data 015.data /

integration/testdata/delta/006.Dockerfile

@@ -0,0 +1,17 @@
+FROM scratch
+
+COPY 001.data /
+COPY 002.data /
+COPY 003.data /
+COPY 004.data /
+COPY 005.data /
+COPY 006.data /
+COPY 007.data /
+COPY 008.data /
+COPY 009.data /
+COPY 010.data /
+COPY 011.data /
+COPY 012.data /
+COPY 013.data /
+COPY 014.data /
+COPY 015.data /

integration/testdata/delta/007.Dockerfile

@@ -0,0 +1,17 @@
+FROM scratch
+
+COPY 002.data /
+COPY 008.data /
+COPY 003.data /
+COPY 005.data /
+COPY 015.data /
+COPY 001.data /
+COPY 011.data /
+COPY 006.data /
+COPY 009.data /
+COPY 007.data /
+COPY 014.data /
+COPY 010.data /
+COPY 012.data /
+COPY 013.data /
+COPY 004.data /

integration/testdata/delta/008.Dockerfile

@@ -0,0 +1,4 @@
+FROM scratch
+
+COPY 001.data 002.data 003.data 004.data 005.data 006.data 007.data 008.data \
+    009.data 010.data 011.data 012.data 013.data 014.data 015.data /

integration/testdata/delta/009.Dockerfile

@@ -0,0 +1,4 @@
+FROM scratch
+
+COPY 000.data 100.data 006.data 009.data /
+

integration/testdata/delta/010.Dockerfile

@@ -0,0 +1,3 @@
+FROM 127.0.0.1:5000/009
+
+COPY 004.data 009.data 011.data /

integration/testdata/delta/011.Dockerfile

@@ -0,0 +1,3 @@
+FROM 127.0.0.1:5000/009
+
+COPY 005.data 009.data 011.data /

integration/testdata/delta/012.Dockerfile

@@ -0,0 +1,3 @@
+FROM scratch
+
+COPY 000.data 101.data 006.data 009.data /

integration/testdata/delta/013.Dockerfile

@@ -0,0 +1,3 @@
+FROM 127.0.0.1:5000/012
+
+COPY 004.data 009.data 011.data /

integration/testdata/delta/README.md

@@ -0,0 +1,34 @@
+# Test data for deltas
+
+## Data files
+
+The data files we use to populate our test Docker images are divided in two
+groups. The first group contains random data:
+
+* `000.data`: An empty file.
+* `001.data`: Another empty file.
+* `002.data`: 256 bytes of random data.
+* `003.data`: 333 bytes of random data.
+* `004.data`: 1024 bytes of random data.
+* `005.data`: 1024 bytes of random data (different from 003).
+* `006.data`: 3307 bytes of random data.
+* `007.data`: 8 KiB of random data.
+* `008.data`: 9876 bytes of random data.
+* `009.data`: 10 KiB of random data.
+* `010.data`: 128 KiB of random data.
+* `011.data`: 150000 bytes of random data.
+* `012.data`: 500003 bytes of random data.
+* `013.data`: 1000000 bytes of random data.
+* `014.data`: 1000000 bytes of random data (different from 012).
+* `015.data`: 1 MiB of random data.
+* `016.data`: 1 MiB of random data (different from 015).
+
+The second group is made by concatenating the files from the first group. In the
+descriptions below, we use the _001 + 002 + 010_ notation to mean "the
+concatenation of `001.data`, `002.data`, and `010.data`." We say _3 * 007_ as a
+shortcut to "the contents of `007.data` repeated three times." And we can go
+fancy and use _4 * (002 + 004)_ to mean "the concatenation of `002.data` and
+`004.data` repeated four times."
+
+* `100.data`: 015 + 013 + 012 + 014 + 010.
+* `101.data`: 016 + 013 + 012 + 014 + 010.