Skip to content
This repository has been archived by the owner. It is now read-only.

Using TLS gives "TLS handshake error ... EOF" in logs #1094

Closed
ahmetb opened this issue Jul 28, 2015 · 6 comments
Closed

Using TLS gives "TLS handshake error ... EOF" in logs #1094

ahmetb opened this issue Jul 28, 2015 · 6 comments

Comments

@ahmetb
Copy link
Contributor

@ahmetb ahmetb commented Jul 28, 2015

question

I am using the same TLS certificates <ca.pem,cert.pem,key.pem> in all Docker engines serving as swarm nodes and all containers running the swarm manager.

Swarm cluster functions just fine, I can create containers that gets sprayed across nodes. However, I'm seeing errors like the following in the swarm manage logs:

http: TLS handshake error from 168.63.129.16:57546: EOF

I realized this IP address (168.63.129.16) is the default nameserver on the virtual machine I provisioned on the cloud (Azure). It's listed in the /etc/resolv.conf as:

nameserver 168.63.129.16
search drs3vjatj2puvj5qgtdy5v0l0d.dx.internal.cloudapp.net

Looks harmless, but any ideas where these errors might be coming from (I could not get the relationship between TLS and DNS)? More details below.


The engine on the nodes are running as:

/usr/bin/docker -d --daemon -H=unix:// --tls --tlscacert=/etc/docker/ca.pem \
  --tlscert=/etc/docker/cert.pem --tlskey=/etc/docker/key.pem \
  -H=0.0.0.0:2375

and the swarm managers are running as:

"Path": "/swarm",
"Args": [
    "--debug",
    "manage",
    "--tls",
    "--tlscacert",
    "/etc/docker/ca.pem",
    "--tlscert",
    "/etc/docker/cert.pem",
    "--tlskey",
    "/etc/docker/key.pem",
    "--replication",
    "--advertise",
    "10.0.0.4:2376",
    "consul://10.0.0.4:8500,10.0.0.5:8500,10.0.0.6:8500/nodes"
]

logs from swarm manager:

time="2015-07-28T05:10:24Z" level=info msg="Listening for HTTP" addr=":2375" proto=tcp
time="2015-07-28T05:10:24Z" level=info msg="Cluster leadership lost"
time="2015-07-28T05:10:24Z" level=debug msg="Watch triggered with 3 nodes" discovery=consul
time="2015-07-28T05:10:24Z" level=info msg="New leader elected: 10.0.0.4:2376"
time="2015-07-28T05:10:25Z" level=debug msg="Updated engine state" id="46NU:2WOU:S2EH:7XQO:657Z:FX3O:X46Z:C5FJ:XNUB:BRSG:QBOC:XMBS" name=swarm-node-1
time="2015-07-28T05:10:25Z" level=debug msg="Updated engine state" id="GZVP:DZXB:3LFW:DCIJ:4UNI:M4KV:TPXW:C24K:3PKK:H2EQ:VA3H:GUMH" name=swarm-node-0
time="2015-07-28T05:10:25Z" level=debug msg="Updated engine state" id="U4E3:W4MX:Y24S:PNZ2:6ORY:PTTI:PWGM:GWMF:4NXD:JEOC:LGPS:LZCF" name=swarm-node-2
time="2015-07-28T05:10:25Z" level=info msg="Registered Engine swarm-node-0 at 192.168.0.4:2375"
time="2015-07-28T05:10:25Z" level=info msg="Registered Engine swarm-node-1 at 192.168.0.5:2375"
time="2015-07-28T05:10:25Z" level=info msg="Registered Engine swarm-node-2 at 192.168.0.6:2375"
time="2015-07-28T05:10:35Z" level=debug msg="Watch triggered with 3 nodes" discovery=consul
2015/07/28 05:10:35 http: TLS handshake error from 168.63.129.16:61480: EOF
time="2015-07-28T05:10:41Z" level=debug msg="Watch triggered with 3 nodes" discovery=consul
time="2015-07-28T05:10:42Z" level=debug msg="Watch triggered with 3 nodes" discovery=consul
2015/07/28 05:10:45 http: TLS handshake error from 168.63.129.16:61545: EOF
time="2015-07-28T05:10:55Z" level=debug msg="Updated engine state" id="U4E3:W4MX:Y24S:PNZ2:6ORY:PTTI:PWGM:GWMF:4NXD:JEOC:LGPS:LZCF" name=swarm-node-2
time="2015-07-28T05:10:55Z" level=debug msg="Updated engine state" id="46NU:2WOU:S2EH:7XQO:657Z:FX3O:X46Z:C5FJ:XNUB:BRSG:QBOC:XMBS" name=swarm-node-1
time="2015-07-28T05:10:55Z" level=debug msg="Updated engine state" id="GZVP:DZXB:3LFW:DCIJ:4UNI:M4KV:TPXW:C24K:3PKK:H2EQ:VA3H:GUMH" name=swarm-node-0
2015/07/28 05:10:55 http: TLS handshake error from 168.63.129.16:61572: EOF
time="2015-07-28T05:10:56Z" level=debug msg="Watch triggered with 3 nodes" discovery=consul
time="2015-07-28T05:11:02Z" level=debug msg="Watch triggered with 3 nodes" discovery=consul
time="2015-07-28T05:11:02Z" level=debug msg="Watch triggered with 3 nodes" discovery=consul
2015/07/28 05:11:05 http: TLS handshake error from 168.63.129.16:61630: EOF
2015/07/28 05:11:15 http: TLS handshake error from 168.63.129.16:61694: EOF
time="2015-07-28T05:11:17Z" level=debug msg="Watch triggered with 3 nodes" discovery=consul
time="2015-07-28T05:11:22Z" level=debug msg="Watch triggered with 3 nodes" discovery=consul
time="2015-07-28T05:11:23Z" level=debug msg="Watch triggered with 3 nodes" discovery=consul
time="2015-07-28T05:11:25Z" level=debug msg="Updated engine state" id="U4E3:W4MX:Y24S:PNZ2:6ORY:PTTI:PWGM:GWMF:4NXD:JEOC:LGPS:LZCF" name=swarm-node-2
time="2015-07-28T05:11:25Z" level=debug msg="Updated engine state" id="46NU:2WOU:S2EH:7XQO:657Z:FX3O:X46Z:C5FJ:XNUB:BRSG:QBOC:XMBS" name=swarm-node-1
time="2015-07-28T05:11:25Z" level=debug msg="Updated engine state" id="GZVP:DZXB:3LFW:DCIJ:4UNI:M4KV:TPXW:C24K:3PKK:H2EQ:VA3H:GUMH" name=swarm-node-0
2015/07/28 05:11:25 http: TLS handshake error from 168.63.129.16:61728: EOF
2015/07/28 05:11:35 http: TLS handshake error from 168.63.129.16:61785: EOF
time="2015-07-28T05:11:38Z" level=debug msg="Watch triggered with 3 nodes" discovery=consul
time="2015-07-28T05:11:42Z" level=debug msg="Watch triggered with 3 nodes" discovery=consul
@ahmetb ahmetb changed the title Using same TLS certs on all nodes and managers gives TLS handshake error in logs Using TLS gives "TLS handshake error ... EOF" in logs Jul 28, 2015
@aluzzardi aluzzardi added this to the 0.5.0 milestone Aug 5, 2015
@aluzzardi aluzzardi self-assigned this Aug 5, 2015
@aluzzardi aluzzardi removed this from the 1.0.0 milestone Nov 27, 2015
@aluzzardi aluzzardi added this to the 1.1.0 milestone Nov 27, 2015
@aluzzardi aluzzardi added this to the 1.1.0 milestone Nov 27, 2015
@aluzzardi aluzzardi removed this from the 1.0.0 milestone Nov 27, 2015
@vieux vieux removed this from the 1.1.0 milestone Feb 10, 2016
@vieux vieux added this to the 1.2.0 milestone Feb 10, 2016
@vieux vieux added this to the 1.2.0 milestone Feb 10, 2016
@vieux vieux removed this from the 1.1.0 milestone Feb 10, 2016
@jotunskij
Copy link

@jotunskij jotunskij commented Mar 22, 2016

+1

@amitshukla amitshukla added this to the 1.3.0 milestone Mar 23, 2016
@amitshukla amitshukla removed this from the 1.2.0 milestone Mar 23, 2016
@mattgiles
Copy link

@mattgiles mattgiles commented Apr 29, 2016

+1

@ahmetb
Copy link
Contributor Author

@ahmetb ahmetb commented Jun 29, 2016

Not valid anymore with new Swarm.

@ahmetb ahmetb closed this Jun 29, 2016
@ghost
Copy link

@ghost ghost commented Aug 4, 2017

+1

@Levinr
Copy link

@Levinr Levinr commented Aug 16, 2017

I am also experiencing the same error

@shibug
Copy link

@shibug shibug commented Sep 5, 2017

I was also experiencing the same issue. It was because the TCP health probes configured on Azure Loadbalancer was probing swarm port 2376 secured by TLS. Even though Azure LB considers it a successful probe, Swarm logs it as a handshake failure because it was a TCP probe.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
8 participants