salt-minion fails with IPv6

[GevatterGaul]

Description of Issue/Question

When I use a domain name that only resolves to a IPv6 address for the master address in the minion config the salt-minion will exit with an error code. With an IPv4 domain, everything works as expected.

Feb 12 01:21:02 mail systemd[1]: Starting The Salt Minion...
Feb 12 01:21:02 mail systemd[1]: Started The Salt Minion.
Feb 12 01:21:04 mail salt-minion[12340]: Permission denied (src/tcp_connecter.cpp:344)
Feb 12 01:21:04 mail systemd[1]: salt-minion.service: Main process exited, code=exited, status=250/n/a
Feb 12 01:21:04 mail systemd[1]: salt-minion.service: Unit entered failed state.
Feb 12 01:21:04 mail systemd[1]: salt-minion.service: Failed with result 'exit-code'.

Setup

Default salt-minion config, except the master entry and ipv6 switch:

master: ipv6-only-domain.duckdns.org
ipv6: True

Steps to Reproduce Issue

Install a current salt-minion and change the master to a ipv6 only domain name. Then restart the salt-minion.

Versions Report

Salt Version:
           Salt: 2016.11.2

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.4.2
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.3
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
         Python: 2.7.12 (default, Nov 19 2016, 06:48:10)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4

System Versions:
           dist: Ubuntu 16.04 xenial
        machine: x86_64
        release: 4.4.0-62-generic
         system: Linux
        version: Ubuntu 16.04 xenial

[gtmanfred]

Can you provide the debug output of the salt minion when it is starting?

I have been unable to get the minion to connect to a master over ipv6, and have been unsuccessful. I just want to compare what you are seeing in debug to my logs.

Thanks,
Daniel

[GevatterGaul]

Here is the debug output:

Feb 15 23:30:24 mail systemd[1]: Starting The Salt Minion...
Feb 15 23:30:24 mail systemd[1]: Started The Salt Minion.
Feb 15 23:30:24 mail salt-minion[26521]: [DEBUG   ] Configuration file path: /etc/salt/minion
Feb 15 23:30:24 mail salt-minion[26521]: [WARNING ] Insecure logging configuration detected! Sensitive data may be logged.
Feb 15 23:30:24 mail salt-minion[26521]: [INFO    ] Setting up the Salt Minion "mail"
Feb 15 23:30:24 mail salt-minion[26521]: [DEBUG   ] Created pidfile: /var/run/salt-minion.pid
Feb 15 23:30:24 mail salt-minion[26521]: [INFO    ] Starting up the Salt Minion
Feb 15 23:30:24 mail salt-minion[26521]: [DEBUG   ] AsyncEventPublisher PUB socket URI: /var/run/salt/minion/minion_event_00d8d3f117_pub.ipc
Feb 15 23:30:24 mail salt-minion[26521]: [DEBUG   ] AsyncEventPublisher PULL socket URI: /var/run/salt/minion/minion_event_00d8d3f117_pull.ipc
Feb 15 23:30:24 mail salt-minion[26521]: [INFO    ] Starting pull socket on /var/run/salt/minion/minion_event_00d8d3f117_pull.ipc
Feb 15 23:30:24 mail salt-minion[26521]: [DEBUG   ] SaltEvent PUB socket URI: /var/run/salt/minion/minion_event_00d8d3f117_pub.ipc
Feb 15 23:30:24 mail salt-minion[26521]: [DEBUG   ] SaltEvent PULL socket URI: /var/run/salt/minion/minion_event_00d8d3f117_pull.ipc
Feb 15 23:30:24 mail salt-minion[26521]: [DEBUG   ] Initializing new IPCClient for path: /var/run/salt/minion/minion_event_00d8d3f117_pub.ipc
Feb 15 23:30:24 mail salt-minion[26521]: [DEBUG   ] Reading configuration from /etc/salt/minion
Feb 15 23:30:25 mail salt-minion[26521]: [DEBUG   ] Including configuration from '/etc/salt/minion.d/_schedule.conf'
Feb 15 23:30:25 mail salt-minion[26521]: [DEBUG   ] Reading configuration from /etc/salt/minion.d/_schedule.conf
Feb 15 23:30:25 mail salt-minion[26521]: [DEBUG   ] Please install 'virt-what' to improve results of the 'virtual' grain.
Feb 15 23:30:25 mail salt-minion[26521]: [INFO    ] Creating minion process manager
Feb 15 23:30:25 mail salt-minion[26521]: [DEBUG   ] Process Manager starting!
Feb 15 23:30:25 mail salt-minion[26521]: [DEBUG   ] Process Manager starting!
Feb 15 23:30:25 mail salt-minion[26521]: [DEBUG   ] Connecting to master. Attempt 1 of 1
Feb 15 23:30:25 mail salt-minion[26521]: [DEBUG   ] Initializing new AsyncAuth for ('/etc/salt/pki/minion', 'mail', 'tcp://[2003:XXXXXX]:4506')
Feb 15 23:30:25 mail salt-minion[26521]: [DEBUG   ] Generated random reconnect delay between '1000ms' and '11000ms' (6720)
Feb 15 23:30:25 mail salt-minion[26521]: [DEBUG   ] Setting zmq_reconnect_ivl to '6720ms'
Feb 15 23:30:25 mail salt-minion[26521]: [DEBUG   ] Setting zmq_reconnect_ivl_max to '11000ms'
Feb 15 23:30:25 mail salt-minion[26521]: [DEBUG   ] Initializing new AsyncZeroMQReqChannel for ('/etc/salt/pki/minion', 'mail', 'tcp://[2003:XXXXXX]:4506', 'clear')
Feb 15 23:30:26 mail salt-minion[26521]: Permission denied (src/tcp_connecter.cpp:344)
Feb 15 23:30:27 mail systemd[1]: salt-minion.service: Main process exited, code=exited, status=250/n/a
Feb 15 23:30:27 mail systemd[1]: salt-minion.service: Unit entered failed state.
Feb 15 23:30:27 mail systemd[1]: salt-minion.service: Failed with result 'exit-code'.

[rallytime]

Hi @GevatterGaul - I was not able to reproduce your error specifically here, but I was able to reproduce some general issues with connecting via IPv6. The status of IPv6 has changed a bit with the addition of #39289, and then my fix goes on top of that with #39766.

Are you in a position to give the head of the 2016.3 branch a try, once my fix goes in? I want to make sure we have this cleaned up for you since we weren't able to reproduce your specific error.

EDIT: Or, if the head of the 2016.11 branch is better (since I see you were testing this with 2016.11.2), then my fix will be merged forward in the next couple of days once my fix goes into the 2016.3 branch.