ps.top raises ValueError "too many values to unpack" when psutil > 4.1.0

[rodoyle]

Description of Issue/Question

1. salt-call pip.install psutil
   output: installs psutil 4.3.0 (current stable release)
2. salt-call ps.top
   output:

Traceback (most recent call last):
  File "/usr/bin/salt-call", line 11, in <module>
    salt_call()
  File "/usr/lib/python2.7/dist-packages/salt/scripts.py", line 345, in salt_call
    client.run()
  File "/usr/lib/python2.7/dist-packages/salt/cli/call.py", line 58, in run
    caller.run()
  File "/usr/lib/python2.7/dist-packages/salt/cli/caller.py", line 134, in run
    ret = self.call()
  File "/usr/lib/python2.7/dist-packages/salt/cli/caller.py", line 197, in call
    ret['return'] = func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/salt/modules/ps.py", line 135, in top
    user, system = process.cpu_times()

Investigating this on the minion:

>>> process = psutil.Process(1)
>>> process.cpu_times()
pcputimes(user=0.56, system=2.52, children_user=1088.18, children_system=73.1)

psutil.Process.cpu_times() now returns more than expected two values on Linux but not windows and OS X according to the documentation.

Salt documentation does not mention a requirement for psutil <=4.10 and the module does not handle the newer return value. Searching the issues for 'psutil' turns up several compatibility issues elsewhere, but none in the ps.top module. It is possible this issue is part of a larger cross-platform compatibility problem in the ps module.

Setup

(Please provide relevant configs and/or SLS files (Be sure to remove sensitive info).)

• None, purely built in modules

psutil should be version 4.3.0

In 4.1.0 new fields were added, "Changed in version 4.1.0: return two extra fields: children_user and children_system." http://pythonhosted.org/psutil/

Steps to Reproduce Issue

(Include debug logs if possible and relevant.)

1. install default psutil from pip on minion with ubuntu 14.04 LTS
2. use built in ps.util module from either locally or remotely

Versions Report

(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)

Salt Version:
Salt: 2016.3.1 (as on master)

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 1.5
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.7.2
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 0.9.1
   msgpack-pure: Not Installed
 msgpack-python: 0.3.0
   mysql-python: 1.2.3
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
         Python: 2.7.6 (default, Jun 22 2015, 17:58:13)
   python-gnupg: Not Installed
         PyYAML: 3.10
          PyZMQ: 14.0.1
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5

System Versions:
           dist: Ubuntu 14.04 trusty
        machine: x86_64
        release: 3.19.0-61-generic
         system: Linux
        version: Ubuntu 14.04 trusty

[Ch3LL]

@rodoyle I am able to replicate this behavior. Looks like we need to add some logic to support psutils 4.1 and above. For anyone looking for a workaround that comes across this issue you can simple run pip install psutil==4.0.0. I confirmed it works with that version. Thanks

[kevinquinnyo]

Looks like this was fixed here ea67120 with this PR #32182

[rallytime]

@kevinquinnyo Good find - we should backport that fix to 2016.3. I've labeled it as such.

[rallytime]

@rodoyle This should be fixed in develop already, which I have now back-ported to 2016.3 with #34318. Can you give that a try?

[btbroot]

Neither psutil 4.0.0 or 3.0.0 work for a Windows minion smoothly (ps.pgrep return null, ps.get_users throws an exception)

[Ch3LL]

@btbroot would you mind opening a seperate issue that includes the exception and more details