pkg module and state not working in Arch Linux #37044

githubcdr · 2016-10-17T13:32:26Z

Description of Issue/Question

The pkg state and module won't work on latest Arch Linux. This might be related to an Arch upgrade, since it was working before in current version.

Setup

Latest Arch Linux x64, running salt-zmq 2016.3.3

Steps to Reproduce Issue

salt-call pkg.version bash
'pkg.version' is not available.

and

     Comment: State 'pkg.uptodate' was not found in SLS 'maintenance.linux.arch'
              Reason: 'pkg' __virtual__ returned False

The module is loaded and the docs confirm this.

salt-call sys.doc pkg

Versions Report

Salt Version:
           Salt: 2016.3.3

Dependency Versions:
           cffi: 1.8.3
       cherrypy: 8.1.0
       dateutil: 2.5.3
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: 1.6.5
         Jinja2: 2.8
        libgit2: 0.24.0
        libnacl: 1.5.0
       M2Crypto: 0.24.0
           Mako: Not Installed
   msgpack-pure: 0.1.3
 msgpack-python: 0.4.8
   mysql-python: Not Installed
      pycparser: 2.14
       pycrypto: 2.6.1
         pygit2: 0.24.1
         Python: 2.7.12 (default, Jun 28 2016, 08:31:05)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 16.0.0
           RAET: Not Installed
          smmap: 0.9.0
        timelib: 0.2.4
        Tornado: 4.4.2
            ZMQ: 4.1.5

System Versions:
           dist:
        machine: x86_64
        release: 4.7.6-1-ARCH
         system: Linux
        version: Not Installed

The text was updated successfully, but these errors were encountered:

damon-atkins · 2016-10-17T13:40:08Z

Given dist: is blank and version: Not Installed, it looks like Salt can not work out what OS its running on, and therefore does not have a package manager for it. Have a look at salts OS detection code.

githubcdr · 2016-10-17T19:13:38Z

Thanks for pointing that out, I don't have the skills to debug this but I think I must look in grains/core.py, is this correct?

This is confusing since the grains are set correctly according to grains.get os etc.

Ch3LL · 2016-10-17T21:53:25Z

Also to mention you should be able to work around this by specifying the following:

providers:
  pkg: apt

But in your instance replace pkg with the name of the package manager you are using.

Also which exact version of arch are you using? Can you paste a version here? And I will see if I can replicate the behavior. thanks

githubcdr · 2016-10-18T10:02:18Z

Hi @Ch3LL ,

Thanks for the feedback, after manually adding that line things are working again, there is something wrong with the os detection, this issue happens on 3 other servers too.

providers:
  pkg: pacman

zsh 4724 # salt-call pkg.version bash
local:
    4.3.046-1

Arch version

zsh 4725 # cat /etc/os-release
NAME="Arch Linux"
ID=arch
PRETTY_NAME="Arch Linux"
ANSI_COLOR="0;36"
HOME_URL="https://www.archlinux.org/"
SUPPORT_URL="https://bbs.archlinux.org/"
BUG_REPORT_URL="https://bugs.archlinux.org/"

I rolled back updated packages (glib2, python2-setuptools, python2-pyparsing, python2-psutil) to see if things recovered, but no show.

As damon-atkins pointed out, there is an issue on Arch notable in the empty --versions output

damon-atkins · 2016-10-18T12:35:07Z

Try this please

$ which python
/usr/bin/python
$ python
Python 2.7.10 (default, Sep  8 2015, 17:36:20)
[GCC 5.1.1 20150618 (Red Hat 5.1.1-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> print platform.dist()
('fedora', '23', 'Twenty Three')
>>> print platform.linux_distribution()
('Fedora', '23', 'Twenty Three')
>>> print platform.mac_ver()
('', ('', '', ''), '')
>>> print platform.win32_ver()
('', '', '', '')

It's bits of the code from /usr/lib/python2.7/site-packages/salt/version.py

githubcdr · 2016-10-18T12:39:54Z

Bingo?

zsh 4733 # python2
Python 2.7.12 (default, Jun 28 2016, 08:31:05)
[GCC 6.1.1 20160602] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> print platform.dist()
('', '', '')
>>>

damon-atkins · 2016-10-18T12:52:11Z

Need you to do all of the above esp.

print platform.dist()
print platform.linux_distribution()

Python 2.7.10 (default, Sep  8 2015, 17:36:20)
[GCC 5.1.1 20150618 (Red Hat 5.1.1-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import salt.version
>>> b=salt.version.system_information()
>>> for h in b:
>>>   print h

('system', 'Linux')
('dist', 'fedora 23 Twenty Three')
('release', '4.4.3-300.fc23.armv7hl')
('machine', 'armv7l')
('version', 'Fedora 23 Twenty Three')
>>>
>>>

/usr/lib/python2.7/platform.py comes with python i.e. your OS vendor. I suspect platform.py has not change but something calls or reads has changed.

You can also try
strace python -m platform -c 'plaform.linux_distribution()' 2>&1 | fgrep 'open("/etc'

damon-atkins · 2016-10-18T13:09:38Z

Can you post a before and after one of these cat /etc/os-release

githubcdr · 2016-10-18T13:10:53Z

My bad;

zsh 4734 # python2
Python 2.7.12 (default, Jun 28 2016, 08:31:05)
[GCC 6.1.1 20160602] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> print platform.dist()
('', '', '')
>>> print platform.linux_distribution()
('', '', '')
>>> print platform.mac_ver()
('', ('', '', ''), '')
>>> print platform.win32_ver()
('', '', '', '')
>>>

and

('system', 'Linux')
('dist', '  ')
('release', '4.7.6-1-ARCH')
('machine', 'x86_64')
('version', '')

damon-atkins · 2016-10-18T13:14:36Z

You can also try
strace python -m platform -c 'plaform.linux_distribution()' 2>&1 | fgrep 'open("/etc'

githubcdr · 2016-10-18T13:20:57Z

Since Arch runs Python2 and 3 aside I had to change a bit

Python2

zsh 4757 # strace python2 -m platform -c 'plaform.linux_distribution()' 2>&1 | fgrep '/etc'
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
open("/etc", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3
stat("/etc/.installed", 0x7ffdead87620) = -1 ENOENT (No such file or directory)

Please let me know if I can help further, this is getting stranger and stranger.

damon-atkins · 2016-10-18T13:24:58Z

Whats in this /etc/.installed on a box which works.

damon-atkins · 2016-10-18T13:26:10Z

Run this on a box which works
strace python2 -m platform -c 'plaform.linux_distribution()' 2>&1 | fgrep 'open("/etc'

githubcdr · 2016-10-18T13:53:08Z

Hi,

/etc/.installed is not available on any server, a working CentOS server shows this;

strace python2 -m platform -c 'plaform.linux_distribution()' 2>&1 | fgrep 'open("/etc'
open("/etc/ld.so.cache", O_RDONLY)      = 3
open("/etc", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 3
open("/etc/redhat-release", O_RDONLY)   = 3

None of the Arch servers work at the moment.

Inspecting /usr/lib/python2.7/platform.py I noticed no support for Arch, leading me to https://bugs.python.org/issue20454

But this is weird, since it was working last week...

damon-atkins · 2016-10-18T14:13:09Z

I suggest you build a VM with the old version of Arch which was working. And try and work out the difference.

Interesting is the comment in https://bugs.python.org/issue20454

Since the direction we are heading seems to toward deprecating linux_distribution (see http://bugs.python.org/issue1322#msg207427), I think this issue should be closed.

Seems they are telling everyone to use https://pypi.python.org/pypi/distro instead of linux_distribution()
(https://github.com/nir0s/distro)

damon-atkins · 2016-10-18T14:35:56Z

I have raised an issue for linux_distribution() being left out of future version of python.

githubcdr · 2016-10-18T14:59:37Z

Great, weird thing that this worked at all :)

Ch3LL · 2016-10-18T20:59:23Z

hmmm nice find guys. Thanks for all the great investigation work. I'm wondering if your old arch boxes if it had a different python version since that python bug report specifically called out python27 and arch returning empty value for linux_distribution(). Can you check that on your old arch boxes?

Either way we need to make sure we can work around this in salt for arch as well. I will label as a bug. Thanks

githubcdr · 2016-10-19T07:34:31Z

I manage 3 Arch instances with Salt, a x64 and 2 ARM architectures, within 1 week I noticed this bug, looking for changes to my systems I noticed the following packages were upgraded usgins Salt.

[2016-10-10 10:40] [ALPM] upgraded bind-tools (9.10.4.P3-1 -> 9.11.0-1)
[2016-10-10 10:40] [ALPM] upgraded jsoncpp (1.7.6-1 -> 1.7.7-1)
[2016-10-10 10:40] [ALPM] upgraded cmake (3.6.2-1 -> 3.6.2-2.1)
[2016-10-10 10:40] [ALPM] upgraded libsystemd (231-3 -> 231-4)
[2016-10-10 10:40] [ALPM] upgraded linux-raspberrypi (4.4.23-1 -> 4.4.24-1)
[2016-10-10 10:40] [ALPM-SCRIPTLET] >>> Updating module dependencies. Please wait ...
[2016-10-10 10:40] [ALPM] upgraded python-pyparsing (2.1.9-1 -> 2.1.10-1)
[2016-10-10 10:40] [ALPM] upgraded python-setuptools (1:28.1.0-1 -> 1:28.3.0-1)
[2016-10-10 10:40] [ALPM] upgraded python2-ioflo (1.6.3-1 -> 1.6.4-1)
[2016-10-10 10:40] [ALPM] upgraded python2-pyparsing (2.1.9-1 -> 2.1.10-1)
[2016-10-10 10:40] [ALPM] upgraded python2-setuptools (1:28.1.0-1 -> 1:28.3.0-1)
[2016-10-10 10:40] [ALPM] upgraded systemd (231-3 -> 231-4)
[2016-10-10 10:40] [ALPM] upgraded systemd-sysvcompat (231-3 -> 231-4)
[2016-10-10 10:40] [ALPM] transaction completed
[2016-10-10 10:40] [ALPM] running 'udev-hwdb.hook'...
[2016-10-10 10:41] [ALPM] running 'update-mime-database.hook'...
[2016-10-11 13:51] [ALPM] transaction started
[2016-10-11 13:52] [ALPM] upgraded linux-firmware (20160915.c4c07a8-1 -> 20161005.9c71af9-1)
[2016-10-11 13:52] [ALPM] upgraded man-pages (4.07-1 -> 4.08-1)
[2016-10-11 13:52] [ALPM] upgraded wiringpi (97.78b5c32-1 -> 100.b0a60c3-1)
[2016-10-11 13:52] [ALPM] transaction completed
[2016-10-12 10:23] [ALPM] transaction started
[2016-10-12 10:23] [ALPM] upgraded linux-raspberrypi (4.4.24-1 -> 4.4.24-2)
[2016-10-12 10:23] [ALPM-SCRIPTLET] >>> Updating module dependencies. Please wait ...
[2016-10-12 10:23] [ALPM] upgraded raspberrypi-firmware-bootloader (20161004-1 -> 20161011-1)
[2016-10-12 10:23] [ALPM] upgraded raspberrypi-firmware-bootloader-x (20161004-1 -> 20161011-1)
[2016-10-12 10:23] [ALPM] upgraded raspberrypi-firmware-tools (20161004-1 -> 20161011-1)
[2016-10-12 10:23] [ALPM] upgraded raspberrypi-firmware-examples (20161004-1 -> 20161011-1)
[2016-10-12 10:23] [ALPM] upgraded raspberrypi-firmware (20161004-1 -> 20161011-1)

I reverted the python2 updates, restarted salt, but no results. What's strange is that the os grains etc are detected correctly.

Ch3LL · 2016-10-19T15:12:44Z

@githubcdr so i understand correctly this is only occurring on one arch box? not all of them? Is tere differences in python versions between them?

githubcdr · 2016-10-19T19:08:08Z

@Ch3LL no this happens on all Arch boxes, python version is the same.

githubcdr · 2016-10-20T20:13:35Z

An update, after extensive debugging I noticed that running Hubblestack (see hubblestack.io) on an Arch machine causes this problem. After disabling nebula_osquery.py in /var/cache/sat/modules things started working again.

@basepi, as maintener of nebula_osquery.py, can you perhaps explain this behaviour?

stale · 2018-07-13T20:50:29Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

Ch3LL added the info-needed waiting for more info label Oct 17, 2016

Ch3LL added this to the Blocked milestone Oct 17, 2016

Ch3LL added State-Module Bug broken, incorrect, or confusing behavior Core relates to code central or existential to Salt severity-medium 3rd level, incorrect or bad functionality, confusing and lacks a work around P4 Priority 4 and removed info-needed waiting for more info labels Oct 18, 2016

Ch3LL modified the milestones: Approved, Blocked Oct 18, 2016

Ch3LL added the Upstream-Bug is a result of an upstream issue, not in salt label Oct 18, 2016

stale bot added the stale label Jul 13, 2018

stale bot closed this as completed Jul 20, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pkg module and state not working in Arch Linux #37044

pkg module and state not working in Arch Linux #37044

githubcdr commented Oct 17, 2016

damon-atkins commented Oct 17, 2016 •

edited

Loading

githubcdr commented Oct 17, 2016 •

edited

Loading

Ch3LL commented Oct 17, 2016

githubcdr commented Oct 18, 2016

damon-atkins commented Oct 18, 2016 •

edited

Loading

githubcdr commented Oct 18, 2016

damon-atkins commented Oct 18, 2016 •

edited

Loading

damon-atkins commented Oct 18, 2016

githubcdr commented Oct 18, 2016

damon-atkins commented Oct 18, 2016 •

edited

Loading

githubcdr commented Oct 18, 2016

damon-atkins commented Oct 18, 2016

damon-atkins commented Oct 18, 2016

githubcdr commented Oct 18, 2016

damon-atkins commented Oct 18, 2016

damon-atkins commented Oct 18, 2016

githubcdr commented Oct 18, 2016

Ch3LL commented Oct 18, 2016

githubcdr commented Oct 19, 2016

Ch3LL commented Oct 19, 2016

githubcdr commented Oct 19, 2016

githubcdr commented Oct 20, 2016

stale bot commented Jul 13, 2018

pkg module and state not working in Arch Linux #37044

pkg module and state not working in Arch Linux #37044

Comments

githubcdr commented Oct 17, 2016

Description of Issue/Question

Setup

Steps to Reproduce Issue

Versions Report

damon-atkins commented Oct 17, 2016 • edited Loading

githubcdr commented Oct 17, 2016 • edited Loading

Ch3LL commented Oct 17, 2016

githubcdr commented Oct 18, 2016

damon-atkins commented Oct 18, 2016 • edited Loading

githubcdr commented Oct 18, 2016

damon-atkins commented Oct 18, 2016 • edited Loading

damon-atkins commented Oct 18, 2016

githubcdr commented Oct 18, 2016

damon-atkins commented Oct 18, 2016 • edited Loading

githubcdr commented Oct 18, 2016

damon-atkins commented Oct 18, 2016

damon-atkins commented Oct 18, 2016

githubcdr commented Oct 18, 2016

damon-atkins commented Oct 18, 2016

damon-atkins commented Oct 18, 2016

githubcdr commented Oct 18, 2016

Ch3LL commented Oct 18, 2016

githubcdr commented Oct 19, 2016

Ch3LL commented Oct 19, 2016

githubcdr commented Oct 19, 2016

githubcdr commented Oct 20, 2016

stale bot commented Jul 13, 2018

damon-atkins commented Oct 17, 2016 •

edited

Loading

githubcdr commented Oct 17, 2016 •

edited

Loading

damon-atkins commented Oct 18, 2016 •

edited

Loading

damon-atkins commented Oct 18, 2016 •

edited

Loading

damon-atkins commented Oct 18, 2016 •

edited

Loading