New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unresolvable masters in the minions masters list cause minion to raise an error #29116

Closed
johnsocp opened this Issue Nov 23, 2015 · 3 comments

Comments

@johnsocp
Contributor

johnsocp commented Nov 23, 2015

When all three masters have valid A records everything works, even if they A records are pointing to 127.0.0.1 but if I remove an A record completely so the hostname specified in the config doesn't exits at all, the minion raises an error and essentially goes offline even though there are two good masters still in the list.

2015-11-23 05:54:33,283 [salt.scripts     ][INFO    ][59357] Sleeping random_reauth_delay of 3 seconds
2015-11-23 05:54:36,444 [salt.cli.daemons ][INFO    ][59390] Setting up the Salt Minion "minion fqdn"
2015-11-23 05:54:36,689 [salt.cli.daemons ][INFO    ][59390] The salt minion is starting up
2015-11-23 05:54:37,316 [salt.loaded.int.module.cmdmod][INFO    ][59390] Executing command ['date', '+%z'] in directory '/root'
2015-11-23 05:54:37,935 [salt.cli.daemons ][INFO    ][59390] The salt minion is shut down
2015-11-23 05:54:37,936 [salt.scripts     ][ERROR   ][59390] Minion failed to start: 
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/salt/scripts.py", line 81, in minion_process
    minion.start()
  File "/usr/lib/python2.7/site-packages/salt/cli/daemons.py", line 275, in start
    self.minion.tune_in()
  File "/usr/lib/python2.7/site-packages/salt/daemons/flo/__init__.py", line 146, in tune_in
    consolepath=consolepath,
  File "/usr/lib/python2.7/site-packages/ioflo/app/run.py", line 138, in run
    skedder.run()
  File "/usr/lib/python2.7/site-packages/ioflo/base/skedding.py", line 273, in run
    status = tasker.runner.send(tasker.desire)
  File "/usr/lib/python2.7/site-packages/ioflo/base/framing.py", line 581, in makeRunner
    self.segue()
  File "/usr/lib/python2.7/site-packages/ioflo/base/framing.py", line 472, in segue
    if frame.precur(): #transition or cond aux was successful so stop evaluating
  File "/usr/lib/python2.7/site-packages/ioflo/base/framing.py", line 1302, in precur
    if act():
  File "/usr/lib/python2.7/site-packages/ioflo/base/acting.py", line 99, in __call__
    return (self.actor(**self.parms))
  File "/usr/lib/python2.7/site-packages/ioflo/base/acting.py", line 526, in __call__
    return self.action(**kwa)
  File "/usr/lib/python2.7/site-packages/ioflo/base/acting.py", line 971, in action
    framer.enter(enters)
  File "/usr/lib/python2.7/site-packages/ioflo/base/framing.py", line 423, in enter
    frame.enter()
  File "/usr/lib/python2.7/site-packages/ioflo/base/framing.py", line 1244, in enter
    act() #call entryAction
  File "/usr/lib/python2.7/site-packages/ioflo/base/acting.py", line 99, in __call__
    return (self.actor(**self.parms))
  File "/usr/lib/python2.7/site-packages/ioflo/base/acting.py", line 526, in __call__
    return self.action(**kwa)
  File "/usr/lib/python2.7/site-packages/salt/daemons/flo/core.py", line 389, in action
    kind=kinds.applKinds.master))
  File "/usr/lib/python2.7/site-packages/raet/road/estating.py", line 235, in __init__
    super(RemoteEstate, self).__init__(stack, uid=uid, **kwa)
  File "/usr/lib/python2.7/site-packages/raet/road/estating.py", line 62, in __init__
    host = socket.gethostbyname(host)
gaierror: [Errno -2] Name or service not known
2015-11-23 05:54:37,940 [salt.scripts     ][WARNING ][59390] ** Restarting minion **

# cat /etc/salt/minion.d/minion.conf 
transport: raet
master: 
  - master1.mydomainname.com
  - master2.mydomainname.com
  - master3.mydomainname.com
master_shuffle: True
master_type: failover
recon_default: 100
recon_max: 5900
recon_randomize: True
log_file: /var/log/salt/minion
log_level: info
log_level_logfile: info

# /usr/bin/salt-minion --versions
Salt Version:
           Salt: 2015.8.1
 
Dependency Versions:
         Jinja2: 2.8
       M2Crypto: Not Installed
           Mako: Not Installed
         PyYAML: 3.11
          PyZMQ: Not Installed
         Python: 2.7.3 (default, May 18 2012, 22:11:42)
           RAET: 0.6.3
        Tornado: 4.2.1
            ZMQ: Not Installed
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: 1.4.3
        libnacl: 1.4.3
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: Not Installed
         pygit2: Not Installed
   python-gnupg: Not Installed
          smmap: Not Installed
        timelib: Not Installed
 

I checked through the open bug reports and didn't see anything that matched.

@jfindlay jfindlay added this to the Approved milestone Nov 23, 2015

@jfindlay

This comment has been minimized.

Show comment
Hide comment
@jfindlay

jfindlay Nov 23, 2015

Contributor

@johnsocp, thanks for the report.

Contributor

jfindlay commented Nov 23, 2015

@johnsocp, thanks for the report.

@cachedout

This comment has been minimized.

Show comment
Hide comment
@cachedout

cachedout Nov 30, 2015

Contributor

@DmitryKuzmenko Is this something you could take a look at?

Contributor

cachedout commented Nov 30, 2015

@DmitryKuzmenko Is this something you could take a look at?

@DmitryKuzmenko

This comment has been minimized.

Show comment
Hide comment
@DmitryKuzmenko
Contributor

DmitryKuzmenko commented Dec 1, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment