tlsverify option in daemon.json works incorrectly when set to false

[gbolo]

Description
when simply creating a /etc/docker/daemon.json file with only tlsverify set to false, docker will not start because it attempts to look for tls certs/keys that don't exist because they have not been defined. I have also tried setting this option to null and an empty string, both attempts also failed.

I know that this could be fixed by simply omitting the option, however I would like to create a default config with all possible options defined for use with configuration management software.

Steps to reproduce the issue:
1 - create a config file the following content:

{
        "tlsverify": false
}

2 - try to launch docker deamon

systemctl start docker

Describe the results you received:
The following error is logged:

Oct 03 00:48:03 lab4 dockerd[6431]: time="2016-10-03T00:48:03.804869036-04:00" level=fatal msg="Could not load X509 key pair (cert: \"\", key: \"\"): open : no such file or directory"
Oct 03 00:48:03 lab4 systemd[1]: docker.service: main process exited, code=exited, status=1/FAILURE
Oct 03 00:48:03 lab4 systemd[1]: Failed to start Docker Application Container Engine.

Describe the results you expected:
I expected docker to start just fine... why is it looking for certs?

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version:

Client:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:
 OS/Arch:      linux/amd64

Output of docker info:

Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 0
Server Version: 1.12.1
Storage Driver: devicemapper
 Pool Name: docker-8:1-410498-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 10.74 GB
 Backing Filesystem: xfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 11.8 MB
 Data Space Total: 107.4 GB
 Data Space Available: 8.828 GB
 Metadata Space Used: 581.6 kB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.147 GB
 Thin Pool Minimum Free Space: 10.74 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 WARNING: Usage of loopback devices is strongly discouraged for production use. Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.107-RHEL7 (2016-06-09)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: null bridge host overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: seccomp
Kernel Version: 3.10.0-327.22.2.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 3.703 GiB
Name: lab4
ID: G4RX:F5IF:DLDD:B3FG:T3X2:N4LT:T66I:24DT:JPHY:STR3:RD7P:P3PB
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: bridge-nf-call-ip6tables is disabled
Insecure Registries:
 127.0.0.0/8

Additional environment details (AWS, VirtualBox, physical, etc.):
not relevant

[thaJeztah]

This is the intended behavior, but it's confusing (others tripped over this). Setting tlsverify to "false", actually enables TLS, but disables verification of the certificate; basically it's the same as --tls=true --tlsverify=false. The --tls flag makes notion of this, but the docs may need some extra info https://docs.docker.com/engine/reference/commandline/dockerd/#/daemon-socket-option

I think this was reconsidered at some point, but would be a breaking change if we changed this behavior

[gbolo]

Thanks for the explanation @thaJeztah

Ideally, it would be nice to have a configuration file have the ability to define all available options fully populated with defaults instead of just omitting them, since an upgrade of dockerd can cause a change in behavior if a once omitted default value changes between versions. I suppose this issue can be closed if this is the intended behavior.

[justincormack]

Your requirement seems reasonable, but I think there must be a way to do
this now.

[gbolo]

Hi @justincormack

It appears that tlsverify option must be absent from the config unless tls is used. Therefore, it cannot be mentioned in the config (even if set to false or null) without dockerd automatically enabling the tls option (even when the tls option is also set to false or null). Please correct me if I am wrong.

[justincormack]

That seems unfortunate, if tls=false is explicitly set.

[hobochili]

Can we get some traction on this? It is not intuitive that "tlsverify": false overrides "tls": false in the config.

We should keep existing behavior in these two cases (ie. enable TLS, don't verify):

{
  "tls": true,
  "tlsverify": false
}

{
  "tlsverify": false
}

But, unless I am missing something, in this case TLS should not be enabled because it is explicitly set to false:

{
  "tls": false,
  "tlsverify": false
}

[jcheroske]

Any update on this? Anyone know current behavior?

[ddrake17]

+1 - I just hit this and also found this confusing.

The way to get this to work is just to use "tls": false in the daemon config file (and an appropriate hosts entry like a super unsecure "hosts":[tcp://0.0.0.0:2375]) and not specify tlsverify at all. Then on your client you need to specify --tls=false when connecting. I was able to connect using a Linux docker client with a command like docker -H tcp://[ip_address]:2375 --tls=false ps

On my Windows 7 docker tool box client I had to take extra steps to run commands successfully. The environment variable DOCKER_TLS_VERIFY is set to 1 by default which overrode the command line setting (that's probably a bug that comes as an extension of this one). Once I removed the environment variable the same command on Linux above worked on Windows 7.

[websitesca]

Just wanted to write here because we hit this as well on Ubuntu and wanted to explain it a little more clearly. The Ubuntu people have setup systemd stuff that starts Docker in a certain mode. If you look at /etc/systemd/system/multi-user.target.wants/docker.service you'll see this systemd unit has ExecStart=/usr/bin/dockerd -H fd:// meaning Docker starts with the -H fd:// option.

Since Docker authors have made it a feature that Docker fails if you try to configure the same option in daemon.json - Docker fails. If you keep trying to change the hosts option in daemon.json Docker will just keep failing. Removing the option will work, but then you can't configure Docker to start the way you want.

You can see why this makes sense - the Ubuntu people want Docker to start in a secure way. If they make it easy to override, more systems will become compromised. Same with the Docker people.

The best way to fix this is to use the override feature of systemd. All you need to do is drop a file into /etc/systemd/system/docker.service.d/override.conf and ensure you have ExecStart=/usr/bin/dockerd and now you'll be able to configure the hosts option in daemon.json no problem. Then if you're using private networks in aws or something similar, you can protect the Docker daemon accordingly.

[KlavsKlavsen]

I can't get this to work (on Ubuntu 20.04).
I tried simply editing ExecStart to set "--tls=false" and I tried "--tls=false --tlsverify=false" and I tried "--tlsverify false"..

I also tried setting them in /etc/docker/config.json

If set tlsverify at all - it complains "failed to create API server: Could not load X509 key pair (cert: "", key: ""): open : no such file or directory"

If I just do --tls=false, when I then try to run:
docker pull prom/alertmanager

it complains over the certificate of my configured proxy is unknown (which it is - and which is why I wanted to disable tls verification) :(

So it seems impossible to use those tls setting for this ?