Make kind load docker-image only load if the image is not already on the nodes

[bouk]

Currently kind load docker-image always tars up the image and sends it, even if the nodes already contains the exact image. To make image loading faster, Kind should look at the image ID and only load over the image if the nodes don't have it yet.

[BenTheElder]

So the tar loading on the node should more or less skip reading images that are already present, for large enough images skipping the tar on the host after checking may still speed up a bit though.

/kind feature
/priority important-longterm

[aojea]

/assign

[k8s-ci-robot]

@aojea: GitHub didn't allow me to assign the following users: aojea.

Note that only kubernetes-sigs members and repo collaborators can be assigned and that issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

/assign

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[rohansingh]

I'm not sure if this should be a separate issue, but I think kind should also only load layers that are not already present on any given node. Here's an example of that kind of incremental load logic:
https://github.com/bazelbuild/rules_docker/blob/master/container/incremental_load.sh.tpl

I'm using that internally to wrap kind load image-archive and only providing layers that are missing in a single-node cluster, and it is significantly faster when pushing changes.

[rohansingh]

Here's the key insight from that script, by the way:

Docker elides layer reads from the tarball when it already has a copy of the layer with the same basis as it has within the tarball. This means that once we have found the lowest layer in our image of which Docker is unaware we must load all of the remaining layers. So to determine existence, iterate through the layers attempting to load the image without it's tarball. As soon as one fails, break and synthesize a "docker save" tarball of all of the remaining layers.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[aojea]

/remove-lifecycle stale

[BenTheElder]

#382 fixed this.