Pin all pip dependencies for Homu #686

aneeshusa · 2017-06-01T00:03:35Z

New versions of requests that were just released removed the bundled
cacert.pem file, causing Homu to be unable to post comments.
Pin requests and all other Homu dependencies to avoid future problems,
now that we are using upgrade=True to ensure Homu gets updated.

This change is


        Pin all Homu pip dependencies

New versions of requests that were just released removed the bundled `cacert.pem` file, causing Homu to be unable to post comments. Pin requests and all other Homu dependencies to avoid future problems, now that we are using `upgrade=True` to ensure Homu gets updated.

aneeshusa · 2017-06-01T00:11:50Z

Part of #601. Follow-up to #593.

aneeshusa · 2017-06-01T01:01:09Z

r? @larsbergstrom or @edunham
Any deploys without this will break Homu, due to the aforementioned requests update, as I just found out.
(Note that this isn't caught by the saltfs Travis cron builder, since Homu starts fine and only chokes when actually trying to post a comment.)


        Use `safety` to check for vulnerable Python packages

Since we are pinning the Python package dependencies for Homu, we need to make sure to upgrade those dependencies when they have vulnerabilities. Use the `safety` CLI tool from https://pyup.io/safety/ for this.

aneeshusa · 2017-06-01T19:27:42Z

I added a commit to use the safety CLI tool to check for vulnerable packages since we are pinning them. Their freely available database is updated once a month, with more frequent updates available using an API Key with a subscription. Mozilla is listed on their front page, do you know if we have an organizational account?

edunham · 2017-06-05T19:58:22Z

@bors-servo r+

Thanks for doing this, Aneesh!

bors-servo · 2017-06-05T19:58:23Z

📌 Commit 6618e33 has been approved by edunham

bors-servo · 2017-06-05T19:58:26Z

⌛ Testing commit 6618e33 with merge c93be67...


        Auto merge of #686 - aneeshusa:pin-requests-for-homu, r=edunham

Pin all pip dependencies for Homu New versions of requests that were just released removed the bundled `cacert.pem` file, causing Homu to be unable to post comments. Pin requests and all other Homu dependencies to avoid future problems, now that we are using `upgrade=True` to ensure Homu gets updated.  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/saltfs/686)

bors-servo · 2017-06-05T20:11:23Z

☀️ Test successful - status-travis
Approved by: edunham
Pushing c93be67 to master...

jdm · 2017-06-14T13:45:11Z

I ran a highstate with test=True and saw these errors, which look relevant:

From servo-linux2:

----------
          ID: pip
    Function: pkg.installed
      Result: True
     Comment: All specified packages are already installed
     Started: 13:37:55.891571
    Duration: 40.616 ms
     Changes:
----------
          ID: virtualenv
    Function: pip.installed
      Result: False
     Comment: An importable pip module is required but could not be found on your system. This usually means that the system's pip package is not installed properly.
     Started: 13:37:56.049861
    Duration: 0.944 ms
     Changes:
----------
          ID: buildbot-slave-dependencies
    Function: pip.installed
      Result: False
     Comment: An importable pip module is required but could not be found on your system. This usually means that the system's pip package is not installed properly.
              An importable pip module is required but could not be found on your system. This usually means that the system's pip package is not installed properly.
     Started: 13:37:56.143335
    Duration: 0.754 ms
     Changes:
----------
          ID: servo-dependencies
    Function: pip.installed
      Result: False
     Comment: One or more requisite failed: python.virtualenv
     Started:
    Duration:
     Changes:
----------

From servo-mac4:

----------
          ID: virtualenv
    Function: pip.installed
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/local/Cellar/saltstack/2015.5.5_1/libexec/lib/python2.7/site-packages/salt/state.py", line 1561, in call
                  **cdata['kwargs'])
                File "/usr/local/Cellar/saltstack/2015.5.5_1/libexec/lib/python2.7/site-packages/salt/states/pip_state.py", line 535, in installed
                  out = _check_pkg_version_format(pkg)
                File "/usr/local/Cellar/saltstack/2015.5.5_1/libexec/lib/python2.7/site-packages/salt/states/pip_state.py", line 159, in _check_pkg_version_format
                  ret['prefix'] = install_req.req.project_name
              AttributeError: 'Requirement' object has no attribute 'project_name'
     Started: 09:37:19.955936
    Duration: 1.948 ms
     Changes:
----------
          ID: buildbot-slave-dependencies
    Function: pip.installed
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/local/Cellar/saltstack/2015.5.5_1/libexec/lib/python2.7/site-packages/salt/state.py", line 1561, in call
                  **cdata['kwargs'])
                File "/usr/local/Cellar/saltstack/2015.5.5_1/libexec/lib/python2.7/site-packages/salt/states/pip_state.py", line 535, in installed
                  out = _check_pkg_version_format(pkg)
                File "/usr/local/Cellar/saltstack/2015.5.5_1/libexec/lib/python2.7/site-packages/salt/states/pip_state.py", line 159, in _check_pkg_version_format
                  ret['prefix'] = install_req.req.project_name
              AttributeError: 'Requirement' object has no attribute 'project_name'
     Started: 09:37:19.958123
    Duration: 1.301 ms
     Changes:
----------
          ID: servo-dependencies
    Function: pip.installed
      Result: False
     Comment: One or more requisite failed: python.virtualenv
     Started:
    Duration:
     Changes:

jdm · 2017-06-14T13:46:48Z

Why is mac4 using saltstack 2015.5.5_1?

highfive added the S-awaiting-review label Jun 1, 2017

aneeshusa force-pushed the aneeshusa:pin-requests-for-homu branch from f507884 to 5b2b2dd Jun 1, 2017

aneeshusa changed the title ~~Pin requests version for Homu~~ Pin all pip dependencies for Homu Jun 1, 2017

highfive assigned larsbergstrom Jun 1, 2017

highfive assigned edunham and unassigned larsbergstrom Jun 5, 2017

highfive added S-awaiting-merge and removed S-awaiting-review labels Jun 5, 2017

bors-servo merged commit 6618e33 into servo:master Jun 5, 2017
2 checks passed

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

homu Test successful
Details

highfive added S-needs-deploy and removed S-awaiting-merge labels Jun 5, 2017

aneeshusa mentioned this pull request Jun 14, 2017

Update servo-mac4 to Salt 2016.3.3 #701

Open

aneeshusa removed the S-needs-deploy label Jun 14, 2017

servo / saltfs

Pin all pip dependencies for Homu #686

Pin all pip dependencies for Homu #686

aneeshusa commented Jun 1, 2017 •

edited

aneeshusa commented Jun 1, 2017

aneeshusa commented Jun 1, 2017

aneeshusa commented Jun 1, 2017

edunham commented Jun 5, 2017

bors-servo commented Jun 5, 2017

bors-servo commented Jun 5, 2017

bors-servo commented Jun 5, 2017

jdm commented Jun 14, 2017

jdm commented Jun 14, 2017

servo / saltfs

Join GitHub today

Pin all pip dependencies for Homu #686

Pin all pip dependencies for Homu #686

Conversation

aneeshusa commented Jun 1, 2017 • edited

aneeshusa commented Jun 1, 2017

aneeshusa commented Jun 1, 2017

aneeshusa commented Jun 1, 2017

edunham commented Jun 5, 2017

bors-servo commented Jun 5, 2017

bors-servo commented Jun 5, 2017

bors-servo commented Jun 5, 2017

jdm commented Jun 14, 2017

jdm commented Jun 14, 2017

aneeshusa commented Jun 1, 2017 •

edited