implement support for daemon --config-file option

[dweomer]

This is a proposal to simplify the divergence of daemon options persistence across providers by leveraging the new --config-file option. Simply put: consistent DaemonOptionsFile persistence into the location specified by --config-file.

Problem

Prior to Docker 1.10 and the advent of the --config-file daemon option, all --engine-* options passed to docker-machine create are persisted in ways specific to the provisioner (unit files for systemd, rc files for upstart, ... whatever suse seems to be doing). The --config-file option allows for us to generate this file in a consistent way across provisioners.

Additionally, any conflict in daemon CLI options with options found in the location specified by --config-file will prevent the daemon from reloading and/or starting. This means that if you want to leverage the new configuration reloading capabilities of the daemon for nodes provisioned by docker-machine you will need to fiddle with them by hand after creation is complete.

Proposal

Implement first-class support for persisting all daemon options to this new daemon config file. This would mean supporting a new --engine-* option implied by such an entry in commands/create.go:

cli.StringFlag{
    Name:   "engine-config-file",
    Usage:  "Specify Docker Daemon configuration file",
    EnvVar: "ENGINE_CONFIG_FILE",
},

With the default value assumed to be overridden by provisioner implementations. This would simplify the template definitions for generic/upstart, systemd, suse, etc such that they would only contain the pointer to the daemon configuration file. More importantly, this would also for nodes created via docker-machine to be able to leverage the new configuration reloading feature.

Potential Issues

Lack of "Native" Support for Configuration Reloading (or more generally, signalling)

• Implement something like docker-machine signal to send a SIGHUP (by default, overridable via switch) to the daemon to easily capitalize on this new functionality.
• Alternatively, modify the behavior of docker-machine provision to send such a signal when only the config file has been updated.

Divining DockerOptionsDir when --engine-config-file is specified?

Should the value of DockerOptionsDir be overridden to be the base path of the specified file?

Yes, this is a shameless attempt to copy @nathanleclaire's proposal format as embodied by #2516

[nathanleclaire]

I think that ideally what should be done is to migrate the existing --engine-* flags to use the new config file, and pull as much as possible out of the existing service / upstart configuration files.

Yes, this is a shameless attempt to copy @nathanleclaire's proposal format as embodied by #2516

Nothing wrong with that, in fact I prefer it ;P

[dweomer]

The "new config file" meaning the machine compose/declaration WIP?

[nathanleclaire]

The "new config file" meaning the machine compose/declaration WIP?

No, the Docker daemon configuration file introduced in 1.10

[dweomer]

Ah, I see. Yes that would simplify it. This reminds me of something that I neglected to add/discuss: support of docker prior to 1.10. I guess the config file could be translated to the target system either way. Removing all of the old --engine-* flags would, however, break backwards compatibility.

[nathanleclaire]

Yes, unfortunately at this point and time Machine (or any other tool, really) is not good at seamless migrations between different versions of Docker.

[dweomer]

Although I like the idea of being able to replace all --engine-* switches with one --engine-config-file plus its corresponding content, I am worried about the materiel that this forces users to gather before being able to successfully create a machine (assuming the other --engine-* switches were removed). This would seem to me to be an advanced option and regardless that we generate a daemon.json on/for the created machine this is not something that we should force the user to provide up front. This implies at least one piece of materiel per machine beyond the fabled, declaritive YAML format.

Additionally, an implication of being able to specify a local daemon.json means that we would need yet another switch, with EnvVar, to specify the path where such would be stored on the target machine.

Thoughts?

[nathanleclaire]

Yeah I would still really like to convert our existing --engine-env stuff to just generate a config file for the daemon.

[dweomer]

@nathanleclaire This is something I would really like to work on but likely won't get the cycles to until July.

[nathanleclaire]

No worries I might try knocking something together if I get some time.