Plugin tests pull from Hub

[aaronlehmann]

I noticed that the plugin integration tests install plugins from the Hub:

pName             = "tiborvass/sample-volume-plugin"
pTag              = "latest"
pNameWithTag      = pName + ":" + pTag

...

func (s *DockerSuite) TestPluginBasicOps(c *check.C) {
        testRequires(c, DaemonIsLinux, IsAmd64, Network)
        _, _, err := dockerCmdWithError("plugin", "install", "--grant-all-permissions", pNameWithTag)
        c.Assert(err, checker.IsNil)
...

I see a few problems with this:

• Use of the Hub means a service outage or transient network issue will make the test appear flaky. We've had this problem with some pull tests before, and have tried to use a local registry instead wherever possible.
• If the plugin is updated on Hub to an incompatible version, it will break integration tests for older versions of Docker.
• The plugin is installed with full privileges, so a compromise of this Hub account can lead to root access on any machine that runs the integration tests.
• It's a waste of bandwidth to pull the plugins from the internet every time a test which uses them runs.

I think it would be a lot better to bake the plugins into the Docker image and serve them from a local registry.

cc @tiborvass @tonistiigi @vdemeester @anusha-ragunathan

[vdemeester]

@aaronlehmann defenitely agreeing with you. I think we should do the same as we are doing for most of the image used in the integration test, i.e. pre-fetching or creating a plugin (or a bunch of them) so that we don't need to pull them.

[aaronlehmann]

I also noticed that TestDaemonNoSpaceLeftOnDeviceError tries to pull registry:2 from Hub.

[aaronlehmann]

Another similar issue in https://jenkins.dockerproject.org/job/Docker-PRs/41506/console:

18:58:22 FAIL: docker_cli_plugins_logdriver_test.go:10: DockerSuite.TestPluginLogDriver
18:58:22 
18:58:22 docker_cli_plugins_logdriver_test.go:15:
18:58:22     dockerCmd(c, "plugin", "install", pluginName)
18:58:22 /go/src/github.com/docker/docker/pkg/testutil/cmd/command.go:64:
18:58:22     t.Fatalf("at %s:%d - %s", filepath.Base(file), line, err.Error())
18:58:22 ... Error: at cli.go:47 - 
18:58:22 Command:  /go/src/github.com/docker/docker/bundles/17.05.0-ce-rc1/binary-client/docker plugin install cpuguy83/docker-logdriver-test:latest
18:58:22 ExitCode: 1
18:58:22 Error:    exit status 1
18:58:22 Stdout:   latest: Pulling from cpuguy83/docker-logdriver-test
18:58:22 
18:58:22 Stderr:   error pulling image configuration: Get https://dseasb33srnrn.cloudfront.net/registry-v2/docker/registry/v2/blobs/sha256/e0/e0dc5bbec53cf81e83c3063d30dfa8307a96145ccf3abc6ab2d381b80e60abac/data?Expires=1492024702&Signature=b8NgTMrtBSEMLrWVoTTLzY1vhxBh70Fyp7Y~CPwB~GQDDxwZONpULJ8Mwe75YnX3J09NGAnBynuKxpjyEtU04mgrepkInhWay12RvJfoww1gM6NEgB2ceCZGJc29rl8m~UofUqN7LZghUs8g7Rhofx~cRqBfojWArScfIWuSCtQ_&Key-Pair-Id=APKAJECH5M7VWIS5YZ6Q: read tcp 172.17.0.2:51129->52.84.50.134:443: read: connection reset by peer

[vdemeester]

Should be closed by #34085 right ?

[thaJeztah]

I think so; let me close, but happy to reopen if we missed some @aaronlehmann

[cpuguy83]

Well, there's still some more to do, namely any plugins that are actually testing a subsystem are still pulled from hub.