servo-mac1 failing to highstate

[larsbergstrom]

Just the failures of a test=True run listed below:

----------
          ID: virtualenv
    Function: pip.installed
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/local/Cellar/saltstack/2015.5.5_1/libexec/lib/python2.7/site-packages/salt/state.py", line 1561, in call
                  **cdata['kwargs'])
                File "/usr/local/Cellar/saltstack/2015.5.5_1/libexec/lib/python2.7/site-packages/salt/states/pip_state.py", line 535, in installed
                  out = _check_pkg_version_format(pkg)
                File "/usr/local/Cellar/saltstack/2015.5.5_1/libexec/lib/python2.7/site-packages/salt/states/pip_state.py", line 159, in _check_pkg_version_format
                  ret['prefix'] = install_req.req.project_name
              AttributeError: 'Requirement' object has no attribute 'project_name'
     Started: 10:56:00.825157
    Duration: 2.354 ms
     Changes:
----------
          ID: buildbot-slave-dependencies
    Function: pip.installed
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/local/Cellar/saltstack/2015.5.5_1/libexec/lib/python2.7/site-packages/salt/state.py", line 1561, in call
                  **cdata['kwargs'])
                File "/usr/local/Cellar/saltstack/2015.5.5_1/libexec/lib/python2.7/site-packages/salt/states/pip_state.py", line 535, in installed
                  out = _check_pkg_version_format(pkg)
                File "/usr/local/Cellar/saltstack/2015.5.5_1/libexec/lib/python2.7/site-packages/salt/states/pip_state.py", line 159, in _check_pkg_version_format
                  ret['prefix'] = install_req.req.project_name
              AttributeError: 'Requirement' object has no attribute 'project_name'
     Started: 10:56:02.529823
    Duration: 3.522 ms
     Changes:
----------
          ID: servo-dependencies
    Function: pip.installed
      Result: False
     Comment: One or more requisite failed: common.virtualenv
     Started:
    Duration:
     Changes:

This is preventing the buildbot slave service from starting on the machine.

cc @aneeshusa @edunham

[edunham]

I've seen this before on mismatched Salt versions.

root@servo-master1:~# salt --version
salt 2015.5.2 (Lithium)

administrators-Mac-mini:~ administrator$ salt --version
salt 2015.5.5 (Lithium)

The quick fix is to downgrade the Mac's salt version; the right solution to prevent it from happening again is to get Aneesh's salt-in-salt fixes landed so it's easy to keep the master up to date.

[larsbergstrom]

Did the following to downgrade to 2015.5.0 and was able to downgrade. Formula does not exist for 2015.5.2.

administrators-Mac-mini:~ root# su - Administrator
administrators-Mac-mini:~ administrator$ brew unlink saltstack
Unlinking /usr/local/Cellar/saltstack/2015.5.5_1... 25 symlinks removed
administrators-Mac-mini:~ administrator$ brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/a5f0818f263ad815319ceb22262cbc0f8e6ee671/Formula/saltstack.rb
######################################################################## 100.0%
==> Downloading https://homebrew.bintray.com/bottles/saltstack-2015.5.1.yosemite.bottle.tar.gz
######################################################################## 100.0%
==> Pouring saltstack-2015.5.1.yosemite.bottle.tar.gz
==> Caveats
Sample configuration files have been placed in /usr/local/etc/saltstack.
Saltstack will not use these by default.
==> Summary
🍺  /usr/local/Cellar/saltstack/2015.5.1: 2,744 files, 27.3M
administrators-Mac-mini:~ administrator$ salt --version
salt 2015.5.0 (Lithium)
administrators-Mac-mini:~ administrator$ 

[larsbergstrom]

cc @aneeshusa

[larsbergstrom]

Actually, that didn't work either - same error. There seems to exist no compatible version of salt currently up on homebrew to work with that? I'll see what's on the other mac builders...

[larsbergstrom]

Hrm, the other macs are all on head (2016.3.1) and seem to work fine. Just servo-mac1 can't highstate? This is so weird.

[aneeshusa]

Original failure is a dupe of #390. You can try installing Salt from the specific Homebrew URL that's in the install script, or else I will get my salting-the-salt-master branch rebased.

[aneeshusa]

#390 has been fixed.