Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

salt-master startup error when not root and files in /etc/salt/minion.d are listable but not readable #29831

Closed
dr4Ke opened this issue Dec 18, 2015 · 8 comments
Closed

salt-master startup error when not root and files in /etc/salt/minion.d are listable but not readable #29831

dr4Ke opened this issue Dec 18, 2015 · 8 comments
Assignees
@dmurphy18
Labels
Bug broken, incorrect, or confusing behavior Core relates to code central or existential to Salt severity-medium 3rd level, incorrect or bad functionality, confusing and lacks a work around
Milestone
Approved

Comments

@dr4Ke
Copy link
Contributor

dr4Ke commented Dec 18, 2015

I'm using salt-master as an unprivileged user salt. The salt-minion is running as root, or any user beside salt.

When the /etc/salt/minion.d is readable for anyone, but files in it are not readable by user salt, the salt-master fails to start with access denied errors:

IOError: [Errno 13] Permission denied: '/etc/salt/minion.d/00_master.conf'

The salt-master should ignore the minion.d directory, imo.

A workaround is to deny access to that directory to everyone beside the owner, so that the salt-master user can't see these files it can't read.
@dr4Ke
Copy link
Contributor Author

dr4Ke commented Dec 18, 2015

Using 2015.8.3:

Salt Version:
           Salt: 2015.8.3

Dependency Versions:
         Jinja2: unknown
       M2Crypto: Not Installed
           Mako: Not Installed
         PyYAML: 3.11
          PyZMQ: 14.5.0
         Python: 2.6.6 (r266:84292, Jul 23 2015, 15:22:56)
           RAET: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
        libnacl: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
   python-gnupg: Not Installed
          smmap: Not Installed
        timelib: Not Installed

System Versions:
           dist: centos 6.7 Final
        machine: x86_64
        release: 2.6.32-573.12.1.el6.x86_64
         system: CentOS 6.7 Final

@dr4Ke
Copy link
Contributor Author

dr4Ke commented Dec 18, 2015

I can't attach the trace output as a file, so let's include some of it here:

[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/00_user.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/00_user.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_jobs.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_jobs.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_log.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_log.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_roster.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_roster.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_state_output.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_state_output.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/99_fix_slow_states_since_2014.1.0.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/99_fix_slow_states_since_2014.1.0.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: node
[TRACE   ] None of the required configuration sections, 'logstash_udp_handler' and 'logstash_zmq_handler', were found the in the configuration. Not loading the Logstash logging handlers module.
[TRACE   ] The required configuration section, 'fluent_handler', was not found the in the configuration. Not loading the fluent logging handlers module.
[DEBUG   ] Configuration file path: /etc/salt/master
[TRACE   ] Trying pysss.getgrouplist for 'salt'
[TRACE   ] Group list for user 'salt': []
[WARNING ] Insecure logging configuration detected! Sensitive data may be logged.
[INFO    ] Setting up the Salt Master
[DEBUG   ] Loaded master key: /etc/salt/pki/master/master.pem
[INFO    ] Preparing the salt key for local communication
[DEBUG   ] Removing stale keyfile: /var/cache/salt/master/.salt_key
[DEBUG   ] Created pidfile: /var/run/salt-master.pid
[INFO    ] The salt master is starting up
[DEBUG   ] LazyLoaded roots.envs
[DEBUG   ] Could not LazyLoad roots.init
[INFO    ] salt-master is starting as user 'salt'
[INFO    ] Current values for max open files soft/hard setting: 1024/4096
[INFO    ] The value for the 'max_open_files' setting, 100000, is higher than what the user running salt is allowed to raise to, 4096. Defaulting to 4096.
[INFO    ] Raising max open files value to 4096
[INFO    ] New values for max open files soft/hard values: 4096/4096
[INFO    ] Creating master process manager
[INFO    ] Creating master maintenance process
[DEBUG   ] Started 'salt.transport.zeromq.<type 'instancemethod'>._publish_daemon' with pid 30794
[INFO    ] Creating master event publisher process
[INFO    ] Starting the Salt Publisher on tcp://0.0.0.0:4505
[INFO    ] Starting the Salt Puller on ipc:///var/run/salt/master/publish_pull.ipc
[DEBUG   ] Started 'salt.utils.event.<type 'type'>.EventPublisher' with pid 30797
[DEBUG   ] Started 'salt.master.<type 'type'>.Maintenance' with pid 30800
[INFO    ] Creating master publisher process
[INFO    ] Creating master request server process
[DEBUG   ] Started 'salt.master.<type 'instancemethod'>.run_reqserver' with pid 30801
[DEBUG   ] Error loading runners.nacl: libnacl import error, perhaps missing python libnacl package
[DEBUG   ] Started 'salt.transport.zeromq.<type 'instancemethod'>.zmq_device' with pid 30802
[DEBUG   ] Started 'salt.master.<type 'type'>.MWorker' with pid 30803
[INFO    ] Setting up the master communication server
[DEBUG   ] Started 'salt.master.<type 'type'>.MWorker' with pid 30810
[DEBUG   ] MasterEvent PUB socket URI: ipc:///var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: ipc:///var/run/salt/master/master_event_pull.ipc
[DEBUG   ] MasterEvent PUB socket URI: ipc:///var/run/salt/master/master_event_pub.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Started 'salt.master.<type 'type'>.MWorker' with pid 30813
[DEBUG   ] MasterEvent PULL socket URI: ipc:///var/run/salt/master/master_event_pull.ipc
[DEBUG   ] MasterEvent PUB socket URI: ipc:///var/run/salt/master/master_event_pub.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] MasterEvent PULL socket URI: ipc:///var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Started 'salt.master.<type 'type'>.MWorker' with pid 30816
[DEBUG   ] MasterEvent PUB socket URI: ipc:///var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: ipc:///var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Started 'salt.master.<type 'type'>.MWorker' with pid 30819
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] MasterEvent PUB socket URI: ipc:///var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: ipc:///var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Could not LazyLoad timezone.get_offset
[DEBUG   ] Could not LazyLoad config.merge
[DEBUG   ] MasterEvent PUB socket URI: ipc:///var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: ipc:///var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/00_user.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/00_user.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_jobs.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_jobs.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/00_user.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/00_user.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_log.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_log.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_jobs.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_jobs.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_roster.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_roster.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_log.conf'
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_state_output.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_state_output.conf
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_log.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_roster.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_roster.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/99_fix_slow_states_since_2014.1.0.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/99_fix_slow_states_since_2014.1.0.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_state_output.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_state_output.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: node
[DEBUG   ] Including configuration from '/etc/salt/master.d/99_fix_slow_states_since_2014.1.0.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/99_fix_slow_states_since_2014.1.0.conf
[DEBUG   ] Missing configuration file: /home/salt/.saltrc
[DEBUG   ] Including configuration from '/etc/salt/master.d/00_user.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/00_user.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_jobs.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_jobs.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: node
[DEBUG   ] MasterEvent PUB socket URI: ipc:///var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: ipc:///var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Missing configuration file: /home/salt/.saltrc
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_log.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_log.conf
[DEBUG   ] MasterEvent PUB socket URI: ipc:///var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: ipc:///var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_roster.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_roster.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_state_output.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_state_output.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/00_user.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/00_user.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/99_fix_slow_states_since_2014.1.0.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/99_fix_slow_states_since_2014.1.0.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/00_user.conf'
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_jobs.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_jobs.conf
[DEBUG   ] Reading configuration from /etc/salt/master.d/00_user.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_jobs.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_jobs.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_log.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_log.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: node
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_log.conf'
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_roster.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_log.conf
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_roster.conf
[DEBUG   ] Missing configuration file: /home/salt/.saltrc
[DEBUG   ] MasterEvent PUB socket URI: ipc:///var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: ipc:///var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_state_output.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_state_output.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_roster.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_roster.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/10_state_output.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/10_state_output.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/99_fix_slow_states_since_2014.1.0.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/99_fix_slow_states_since_2014.1.0.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: node
[DEBUG   ] Including configuration from '/etc/salt/minion.d/00_master.conf'
[DEBUG   ] Including configuration from '/etc/salt/master.d/99_fix_slow_states_since_2014.1.0.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/99_fix_slow_states_since_2014.1.0.conf
[DEBUG   ] Reading configuration from /etc/salt/minion.d/00_master.conf
Process Maintenance-3:
Traceback (most recent call last):
  File "/usr/lib64/python2.6/multiprocessing/process.py", line 232, in _bootstrap
    self.run()
  File "/usr/lib/python2.6/site-packages/salt/master.py", line 187, in run
    self._post_fork_init()
  File "/usr/lib/python2.6/site-packages/salt/master.py", line 174, in _post_fork_init
    self.search = salt.search.Search(self.opts)
  File "/usr/lib/python2.6/site-packages/salt/search/__init__.py", line 99, in __init__
    matcher=False)
  File "/usr/lib/python2.6/site-packages/salt/minion.py", line 558, in __init__
[DEBUG   ] Missing configuration file: /home/salt/.saltrc
[DEBUG   ] MasterEvent PUB socket URI: ipc:///var/run/salt/master/master_event_pub.ipc
    self.opts = salt.config.minion_config(opts['conf_file'])
  File "/usr/lib/python2.6/site-packages/salt/config.py", line 1592, in minion_config
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: node
    overrides.update(include_config(default_include, path, verbose=False))
  File "/usr/lib/python2.6/site-packages/salt/config.py", line 1514, in include_config
    opts = _read_conf_file(fn_)
  File "/usr/lib/python2.6/site-packages/salt/config.py", line 1374, in _read_conf_file
    with salt.utils.fopen(path, 'r') as conf_file:
  File "/usr/lib/python2.6/site-packages/salt/utils/__init__.py", line 1204, in fopen
[DEBUG   ] MasterEvent PULL socket URI: ipc:///var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Missing configuration file: /home/salt/.saltrc
[DEBUG   ] MasterEvent PUB socket URI: ipc:///var/run/salt/master/master_event_pub.ipc
    fhandle = open(*args, **kwargs)
IOError: [Errno 13] Permission denied: '/etc/salt/minion.d/00_master.conf'
[INFO    ] Process <class 'salt.master.Maintenance'> (30800) died with exit status None, restarting...
[DEBUG   ] MasterEvent PULL socket URI: ipc:///var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/minion.d/00_master.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/00_master.conf
Process MWorker-4:3:
Traceback (most recent call last):
  File "/usr/lib64/python2.6/multiprocessing/process.py", line 232, in _bootstrap
    self.run()
  File "/usr/lib/python2.6/site-packages/salt/master.py", line 760, in run
    self.key,
  File "/usr/lib/python2.6/site-packages/salt/master.py", line 1436, in __init__
    rend=False)
  File "/usr/lib/python2.6/site-packages/salt/minion.py", line 558, in __init__
[DEBUG   ] Including configuration from '/etc/salt/minion.d/00_master.conf'
    self.opts = salt.config.minion_config(opts['conf_file'])
  File "/usr/lib/python2.6/site-packages/salt/config.py", line 1592, in minion_config
    overrides.update(include_config(default_include, path, verbose=False))
  File "/usr/lib/python2.6/site-packages/salt/config.py", line 1514, in include_config
    opts = _read_conf_file(fn_)
  File "/usr/lib/python2.6/site-packages/salt/config.py", line 1374, in _read_conf_file
    with salt.utils.fopen(path, 'r') as conf_file:
  File "/usr/lib/python2.6/site-packages/salt/utils/__init__.py", line 1204, in fopen
    fhandle = open(*args, **kwargs)
IOError: [Errno 13] Permission denied: '/etc/salt/minion.d/00_master.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/00_master.conf
Process MWorker-4:2:
Traceback (most recent call last):
  File "/usr/lib64/python2.6/multiprocessing/process.py", line 232, in _bootstrap
    self.run()
  File "/usr/lib/python2.6/site-packages/salt/master.py", line 760, in run
    self.key,
  File "/usr/lib/python2.6/site-packages/salt/master.py", line 1436, in __init__
    rend=False)
  File "/usr/lib/python2.6/site-packages/salt/minion.py", line 558, in __init__
    self.opts = salt.config.minion_config(opts['conf_file'])
  File "/usr/lib/python2.6/site-packages/salt/config.py", line 1592, in minion_config
    overrides.update(include_config(default_include, path, verbose=False))
  File "/usr/lib/python2.6/site-packages/salt/config.py", line 1514, in include_config
    opts = _read_conf_file(fn_)
  File "/usr/lib/python2.6/site-packages/salt/config.py", line 1374, in _read_conf_file
    with salt.utils.fopen(path, 'r') as conf_file:
  File "/usr/lib/python2.6/site-packages/salt/utils/__init__.py", line 1204, in fopen
    fhandle = open(*args, **kwargs)
IOError: [Errno 13] Permission denied: '/etc/salt/minion.d/00_master.conf'
[DEBUG   ] Including configuration from '/etc/salt/minion.d/00_master.conf'

@jfindlay jfindlay added Bug broken, incorrect, or confusing behavior severity-medium 3rd level, incorrect or bad functionality, confusing and lacks a work around P3 Priority 3 Core relates to code central or existential to Salt labels Dec 18, 2015
@jfindlay jfindlay added this to the Approved milestone Dec 18, 2015
@jfindlay
Copy link
Contributor

@dr4Ke, thanks for the report.

cachedout pushed a commit to cachedout/salt that referenced this issue Dec 29, 2015
Don't load minion opts in a master
b6ca511 
There should be no need to load grains into a master process. This is expensive and time-consuming. Additionally, it attempts to parse the minion config file which should never be necessary for a master.

Closes saltstack#29831
@cachedout cachedout mentioned this issue Dec 29, 2015
Closed
@cachedout
Copy link
Contributor

OK, let's see if #30068 does the trick here.

@cachedout cachedout added the fixed-pls-verify fix is linked, bug author to confirm fix label Dec 29, 2015
@rallytime rallytime removed the fixed-pls-verify fix is linked, bug author to confirm fix label Feb 11, 2016
@stale
Copy link

stale bot commented Mar 23, 2018

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

@stale stale bot added the stale label Mar 23, 2018
@stale stale bot closed this as completed Mar 30, 2018
@Whissi Whissi mentioned this issue Aug 4, 2022
Open
@OrangeDog OrangeDog reopened this Aug 8, 2022
@OrangeDog OrangeDog removed P3 Priority 3 stale labels Aug 8, 2022
@dmurphy18
Copy link
Contributor

@dr4Ke Closing this since very old, and does not have current template tracking metrics.
With Salt 3006.0 and up, salt-master as non-root is fixed.
@OrangeDog It would have been better to open a new issue for this problem rather than resurrecting an old issue, that was using Python 2. Please open a new issue, noting that permissions will revised given @barneysowood PR recently got merged #64194

@dmurphy18 dmurphy18 self-assigned this Aug 22, 2023
@OrangeDog
Copy link
Contributor

@dmurphy18 there already is a new issue: #62428

It is better to keep the oldest version of the issue as the working copy, so information does not get lost and you get a true picture of how long it has been a problem.

Especially when they were closed by stale-bot simply because it took the core team too long to get around to them.

@dmurphy18
Copy link
Contributor

@OrangeDog Normally I would agree with you, but going through cleaning up some of these languishing old issues, Python 2.6 and 2.7. Things have moved on and best to get fresh results with Python 3.
With limited resources, closing and asking to retest if still interested with latest, and the new issues have metrics associated with the template used in filling out which helps keeping track of things better.

And @barneysowood PR should fix the associated issue which should be available very soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dmurphy18 dmurphy18

Labels
Bug broken, incorrect, or confusing behavior Core relates to code central or existential to Salt severity-medium 3rd level, incorrect or bad functionality, confusing and lacks a work around
Projects
None yet
Milestone
Approved
Development

No branches or pull requests
6 participants
@cachedout @dr4Ke @OrangeDog @jfindlay @rallytime @dmurphy18