Salt-Cloud: Authentication failure of type "user" occurred #34124

Horgix · 2016-06-19T22:06:29Z

Summary

salt-cloud, when running against a master and minion in foreground and debug mode, fails to authenticate.

Details

I'm currently trying to run salt (both master and minion) in foreground from sources so I can start playing with the codebase to submlit a PR for #34086 . I installed it following the "Installing Salt for development guidelines", and it's working kinda fine except when trying to get salt-cloud to run.

I'm actually hitting an authentication error on the master, with no idea why and with a standard call to salt with test.ping working perfectly fine.

Note : I'm not hitting this issue with a "normal" setup (Master on Fedora 23, Minion on Archlinux, both in daemon mode launched by systemd from distro packages).

Salt-Master logs

[TRACE   ] IPCServer: Handling connection to address: 
[TRACE   ] Clear payload received with command publish
[WARNING ] Authentication failure of type "user" occurred.

Nothing else when calling salt-cloud.

Salt-Minion logs

Nothing in trace log-level when calling salt-cloud.

Salt-Cloud logs

Before reading the error message : yes, I checked my disk and everything is fine.

$ salt-cloud -c /my_salt_workdir/venv/etc/salt --list-profiles -l trace
/my_salt_workdir/salt-git/salt/config/__init__.py:2466: DeprecationWarning: The term 'provider' is being deprecated in favor of 'driver'. Support for 'provider' will be removed in Salt Nitrogen. Please convert your cloud provider configuration files to use 'driver'.
[ERROR   ] An un-handled exception was caught by salt's global exception handler:
EauthAuthenticationError: Failed to authenticate! This is most likely because this user is not permitted to execute commands, but there is a small possibility that a disk error occurred (check disk/inode usage).
Traceback (most recent call last):
  File "/my_salt_workdir/venv/bin/salt-cloud", line 6, in <module>
    exec(compile(open(__file__).read(), __file__, 'exec'))
  File "/my_salt_workdir/salt-git/scripts/salt-cloud", line 10, in <module>
    salt_cloud()
  File "/my_salt_workdir/salt-git/salt/scripts.py", line 444, in salt_cloud
    client.run()
  File "/my_salt_workdir/salt-git/salt/cloud/cli.py", line 94, in run
    mapper = salt.cloud.Map(self.config)
  File "/my_salt_workdir/salt-git/salt/cloud/__init__.py", line 1666, in __init__
    Cloud.__init__(self, opts)
  File "/my_salt_workdir/salt-git/salt/cloud/__init__.py", line 504, in __init__
    self.__filter_non_working_providers()
  File "/my_salt_workdir/salt-git/salt/cloud/__init__.py", line 1645, in __filter_non_working_providers
    if self.clouds[fun]() is False:
  File "/my_salt_workdir/salt-git/salt/cloud/clouds/lxc.py", line 548, in get_configured_provider
    ret = _salt('test.ping', salt_target=data['target'])
  File "/my_salt_workdir/salt-git/salt/cloud/clouds/lxc.py", line 196, in _salt
    **rkwargs)
  File "/my_salt_workdir/salt-git/salt/client/__init__.py", line 351, in cmd_async
    **kwargs)
  File "/my_salt_workdir/salt-git/salt/client/__init__.py", line 319, in run_job
    return self._check_pub_data(pub_data)
  File "/my_salt_workdir/salt-git/salt/client/__init__.py", line 244, in _check_pub_data
    'Failed to authenticate! This is most likely because this '
EauthAuthenticationError: Failed to authenticate! This is most likely because this user is not permitted to execute commands, but there is a small possibility that a disk error occurred (check disk/inode usage).
Traceback (most recent call last):
  File "/my_salt_workdir/venv/bin/salt-cloud", line 6, in <module>
    exec(compile(open(__file__).read(), __file__, 'exec'))
  File "/my_salt_workdir/salt-git/scripts/salt-cloud", line 10, in <module>
    salt_cloud()
  File "/my_salt_workdir/salt-git/salt/scripts.py", line 444, in salt_cloud
    client.run()
  File "/my_salt_workdir/salt-git/salt/cloud/cli.py", line 94, in run
    mapper = salt.cloud.Map(self.config)
  File "/my_salt_workdir/salt-git/salt/cloud/__init__.py", line 1666, in __init__
    Cloud.__init__(self, opts)
  File "/my_salt_workdir/salt-git/salt/cloud/__init__.py", line 504, in __init__
    self.__filter_non_working_providers()
  File "/my_salt_workdir/salt-git/salt/cloud/__init__.py", line 1645, in __filter_non_working_providers
    if self.clouds[fun]() is False:
  File "/my_salt_workdir/salt-git/salt/cloud/clouds/lxc.py", line 548, in get_configured_provider
    ret = _salt('test.ping', salt_target=data['target'])
  File "/my_salt_workdir/salt-git/salt/cloud/clouds/lxc.py", line 196, in _salt
    **rkwargs)
  File "/my_salt_workdir/salt-git/salt/client/__init__.py", line 351, in cmd_async
    **kwargs)
  File "/my_salt_workdir/salt-git/salt/client/__init__.py", line 319, in run_job
    return self._check_pub_data(pub_data)
  File "/my_salt_workdir/salt-git/salt/client/__init__.py", line 244, in _check_pub_data
    'Failed to authenticate! This is most likely because this '
salt.exceptions.EauthAuthenticationError: Failed to authenticate! This is most likely because this user is not permitted to execute commands, but there is a small possibility that a disk error occurred (check disk/inode usage).

Steps to Reproduce Issue

Install Salt following https://docs.saltstack.com/en/latest/topics/development/hacking.html
Configure Master and Minion to run on the same host
Make sure to have "user: root" in minion, master and cloud configuration
Call salt-cloud

I can provide master, minion and cloud configuration if necessary, but I triple checked that everything was fine, and it's confirmed by the working usual salt calls.

Versions Report

Master and Minion running on Arch Linux, same host
Master and Minion running from HEAD of develop branch
Setup done by following https://docs.saltstack.com/en/latest/topics/development/hacking.html

Salt Master

$ salt-master --versions-report
Salt Version:
           Salt: 2016.3.0-742-g3382bc5

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: 0.24.0
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.7
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
         Python: 2.7.11 (default, Mar 31 2016, 06:18:34)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.3
            ZMQ: 4.1.4

System Versions:
           dist:   
        machine: x86_64
        release: 4.6.2-1-ARCH
         system: Linux
        version: Not Installed

Salt Minion

$ salt-minion --versions-report
Salt Version:
           Salt: 2016.3.0-742-g3382bc5

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: 0.24.0
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.7
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
         Python: 2.7.11 (default, Mar 31 2016, 06:18:34)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.3
            ZMQ: 4.1.4

System Versions:
           dist:   
        machine: x86_64
        release: 4.6.2-1-ARCH
         system: Linux
        version: Not Installed

The text was updated successfully, but these errors were encountered:

Ch3LL · 2016-06-20T19:00:23Z

@Horgix i am not able to replicate this behavior even when using a virtual environment. It looks like there is an extra steps for setup using salt development on arch in those docs.

On Arch Linux, where Python 3 is the default installation of Python, use the virtualenv2 command instead of virtualenv.

Did you do this step during setup?
Also just to ensure when running salt-cloud command are you running it as root? Also if you don't specify user in your cloud/master/minion config do you see a difference?

Horgix · 2016-06-20T19:16:43Z

From Ch3LL:

On Arch Linux, where Python 3 is the default installation of Python, use the virtualenv2 command instead of virtualenv.

Did you do this step during setup?

Yes I did

Also just to ensure when running salt-cloud command are you running it as root?

Yes, I even totally switched to root instead of sudo-ing it.

Also if you don't specify user in your cloud/master/minion config do you see a difference?

I indeed just removed the "user: root" part from the cloud/master/minion config and now it's working ! That's kind of surprising, to say the least. Do you want me to experiment a bit more with that in order to confirm or infirm this bug report or should I just forget it now that it looks like it is working ?

Horgix · 2016-06-28T21:19:15Z

Ping ?

ipmb · 2016-12-20T21:03:34Z

I see this as well in pretty "stock" environment. I do not have user: root defined in my config files.

2016-12-20 08:54:01,773 [salt.cloud.cli   ][ERROR   ][31802] There was a profile error: Failed to authenticate! This is most likely because this user is not permitted to execute commands, but there is a small possibility that a disk error occurred (check disk/inode usage).
2016-12-20 09:46:43,822 [salt.master      ][WARNING ][3738] Authentication failure of type "user" occurred.

The traceback is:

2016-12-20 11:43:20,799 [salt.master      ][WARNING ][4352] Authentication failure of type "user" occurred.
2016-12-20 11:43:20,802 [salt.state       ][ERROR   ][9839] An exception occurred in this state: Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/state.py", line 1744, in call
    **cdata['kwargs'])
  File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1702, in wrapper
    return f(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/salt/states/cloud.py", line 277, in profile
    info = __salt__['cloud.profile'](profile, name, vm_overrides=kwargs, opts=opts)
  File "/usr/lib/python2.7/dist-packages/salt/modules/cloud.py", line 199, in profile_
    info = client.profile(profile, names, vm_overrides=vm_overrides, **kwargs)
  File "/usr/lib/python2.7/dist-packages/salt/cloud/__init__.py", line 349, in profile
    mapper.run_profile(profile, names, vm_overrides=vm_overrides)
  File "/usr/lib/python2.7/dist-packages/salt/cloud/__init__.py", line 1458, in run_profile
    ret[name] = self.create(vm_)
  File "/usr/lib/python2.7/dist-packages/salt/cloud/__init__.py", line 1315, in create
    timeout=self.opts['timeout']
  File "/usr/lib/python2.7/dist-packages/salt/client/__init__.py", line 639, in cmd
    **kwargs)
  File "/usr/lib/python2.7/dist-packages/salt/client/__init__.py", line 337, in run_job
    return self._check_pub_data(pub_data)
  File "/usr/lib/python2.7/dist-packages/salt/client/__init__.py", line 256, in _check_pub_data
    'Failed to authenticate! This is most likely because this '
EauthAuthenticationError: Failed to authenticate! This is most likely because this user is not permitted to execute commands, but there is a small possibility that a disk error occurred (check disk/inode usage).

$ sudo salt --versions
Salt Version:
           Salt: 2016.3.4

Dependency Versions:
           cffi: 1.5.2
       cherrypy: Not Installed
       dateutil: 2.4.2
          gitdb: 0.6.4
      gitpython: 1.0.1
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: 0.24.0
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.3
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: 2.14
       pycrypto: 2.6.1
         pygit2: 0.24.0
         Python: 2.7.12 (default, Nov 19 2016, 06:48:10)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: 0.9.0
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4

System Versions:
           dist: Ubuntu 16.04 xenial
        machine: x86_64
        release: 4.4.0-45-generic
         system: Linux
        version: Ubuntu 16.04 xenial

gtmanfred · 2017-04-14T01:09:40Z

I am going to close this in favor of #40695 I have replicated this issue over there.

Thanks!
Daniel

Ch3LL added cannot-reproduce cannot be replicated with info/context provided info-needed waiting for more info labels Jun 20, 2016

Ch3LL added this to the Blocked milestone Jun 20, 2016

notpeter mentioned this issue Apr 14, 2017

sync_after_install: grains causes salt-cloud to fail #40695

Closed

gtmanfred closed this as completed Apr 14, 2017

gtmanfred added Duplicate Duplicate of another issue or PR - will be closed Bug broken, incorrect, or confusing behavior and removed cannot-reproduce cannot be replicated with info/context provided Duplicate Duplicate of another issue or PR - will be closed labels Apr 14, 2017

gtmanfred modified the milestones: Approved, Blocked Apr 14, 2017

gtmanfred added Duplicate Duplicate of another issue or PR - will be closed and removed info-needed waiting for more info labels Apr 14, 2017

cavepopo mentioned this issue Sep 11, 2019

sync_after_install: grains causes salt-cloud to fail #54452

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Salt-Cloud: Authentication failure of type "user" occurred #34124

Salt-Cloud: Authentication failure of type "user" occurred #34124

Horgix commented Jun 19, 2016 •

edited

Ch3LL commented Jun 20, 2016

Horgix commented Jun 20, 2016

Horgix commented Jun 28, 2016

ipmb commented Dec 20, 2016

gtmanfred commented Apr 14, 2017

Salt-Cloud: Authentication failure of type "user" occurred #34124

Salt-Cloud: Authentication failure of type "user" occurred #34124

Comments

Horgix commented Jun 19, 2016 • edited

Summary

Details

Salt-Master logs

Salt-Minion logs

Salt-Cloud logs

Steps to Reproduce Issue

Versions Report

Salt Master

Salt Minion

Ch3LL commented Jun 20, 2016

Horgix commented Jun 20, 2016

Horgix commented Jun 28, 2016

ipmb commented Dec 20, 2016

gtmanfred commented Apr 14, 2017

Horgix commented Jun 19, 2016 •

edited