Support for external daemon plugins

[ehazlett]

This adds support for external daemon plugins (currently GRPC). This is a light port of the containerd plugin model where plugins can Register as a daemon plugin. These are then initialized at init and can operate directly with the daemon. This is useful, for example, if a plugin needs to operate on an internal system such as the content store that is not accessible via the API.

An example plugin could be something like this:

// local implements the api/server/router/grpc.Backend interface
type local struct {}

func init() {
        plugin.Register(&plugin.PluginRegistration{
                Type: plugin.GRPCPlugin,
                ID:   "foo",
                InitFn: func(ic *plugin.PluginContext) (interface{}, error) {
                        return &local{}, nil
                },
        })
}

Then plugins can simply be included at build time with a blank import:

import (
        ...
        _ "github.com/ehazlett/foo/plugin"
)

[thaJeztah]

Just quickly glancing over, but this allows plugins to register their own endpoints? should those custom endpoints be namespaced (taking the odd case into account that someone uses an authz plug-in or other means to blacklist endpoints)?

Might have to be described somewhere in https://github.com/moby/moby/blob/master/docs/api/version-history.md

[thaJeztah]

just some quick questions after looking at this from my phone 🤗

[ehazlett]

should those custom endpoints be namespaced

Each daemon plugin is namespaced (by default <type>.<id> for example io.docker.grpc.v1.foo). This could be used as a whitelist / blacklist but since these are compile time I'm not sure it's vital. I can see that being more useful if we added runtime loading.

[thaJeztah]

I'm thinking if we should use com.docker... instead of io.docker.... Looking through the codebase, it looks like we use com.docker.. for swarmkit labels, libnetwork options, and some other places, so it might be more consistent.

[andrewhsu]

SGTM

In terms of adding ability to bring in new functionality for the daemon endpoints, this looks like a desirable PR. I'm expecting this does not imply bringing over all of the current HTTP endpoints of the daemon into gRPC.

[AkihiroSuda]

needs rebase