#7630 Adding better error messaging in mach bootstrap for missing virtualenv/pip dependencies

[AnthonyBroadCrawford]

This PR is in reference to #7630

I've added a simple try catch around our use of subprocess.check_all when trying to invoke and use python's

• virtualenv
• pip

Upon failure, I use sys.exit with an error message for the user. Exit seemed appropriate as anything beneath those dependencies will fail to execute and result in a non friendly error message

[highfive]

Thanks for the pull request, and welcome! The Servo team is excited to review your changes, and you should hear from @jdm (or someone else) soon.

[frewsxcv]

So your current solution will indeed warn if the user doesn't have pip or virtualenv installed, though I'm a little worried an OSError will also get raised with some other issue during the os.check_call (network timeout, invalid permissions, corrupted files, etc). What do you think about this idea:

1. Make _get_exec (the function in this the file) return None if both name and default return None when passed to find_executable.
2. In _activate_virtualenv, call _get_exec for both virtualenv (this is already done) and pip and if the result is None, call sys.exit.

[AnthonyBroadCrawford]

@frewsxcv That makes sense, let me take a stab at it.

Additionally, are there unit tests for these scripts? I couldn't find any.

[frewsxcv]

I am not aware of any unit tests for these bootstrapping scripts. That said, if you're considering adding some, I might recommend opening a dedicated GitHub Issue for it so others can chime in their opinions.

[AnthonyBroadCrawford]

@frewsxcv I've made some changes to the code and pushed up to this branch.

I went down the path of expanding the scope on the method _get_exec as you suggested however, while digging into the documentation for subprocess.check_call I realized we are really dealing with three scenarios

1. The referenced executable fails during execution
2. The referenced executable is not found
3. The user doesn't have permissions to execute the executable even when found

Digging into the documentation on subprocess.check_call I found that it will return two types of errors

• subprocess.CalledProcessError when the executable fails during execution (use-case 1 mentioned above)
  -OSError When the process isn't found nor the user has permissions to execute the executable (use-case 2 && 3 from above)

Given that, thought it might be a less invasive change if we expand our exception handling and messaging for the use-cases around our use of check_call.

So, my proposed change is to

1. Catch both exception types subprocess.CalledProcessError and OSError
2. Provide more targeted user friendly error messaging to the user to help steer them to the appropriate resolution.

I'm still 100% open to going down the other path (and to be clear, I did) but it seemed like I was really expanding scope on those functions when this might be less invasive change that helps make the error messages more user friendly when these executables are not installed on their machine.

Thoughts?

[frewsxcv]

^{python/mach_bootstrap.py, line 95 [r2] (raw file):}
This shouldn't be indented, right? Otherwise, the virtual environment will only get activated when it doesn't exist

---

Comments from the review on Reviewable.io

[frewsxcv]

That all seems alright to me. I left a comment on Reviewable before that needs to be addressed

[AnthonyBroadCrawford]

@frewsxcv Fixed the indentation issue. My apologies, still refreshing myself on Python as it has been a while.

[frewsxcv]

@AnthonyBroadCrawford Thanks!

It looks like there is a 'tidy' failure (check the Travis CI failure for more information)

./python/mach_bootstrap.py:93: Line is longer than 120 characters

You should be able to do something like this:

sys.exit("Python virtualenv not found. Please install virtualenv and ensure "
         "permissions prior to running mach.")

[AnthonyBroadCrawford]

@frewsxcv on it.

[AnthonyBroadCrawford]

@frewsxcv Fixed. Thanks for pointing that out.

[frewsxcv]

Looks great! 👍

Can you squash your commits into one? If you need help with this, let me know

[AnthonyBroadCrawford]

@frewsxcv squashed down into a single commit.

[frewsxcv]

@AnthonyBroadCrawford Thanks for your contribution, great work! ✨

@bors-servo r+

[bors-servo]

📌 Commit 2e0e228 has been approved by frewsxcv

[bors-servo]

⌛ Testing commit 2e0e228 with merge a6208f0...

[bors-servo]

💔 Test failed - linux-dev

[AnthonyBroadCrawford]

@frewsxcv, @jdm hmmm, I'm not familiar with what I'm looking at in the "homu" log files. Pointing me in the correct direction would be super helpful.

[frewsxcv]

@AnthonyBroadCrawford Don't worry, there's an issue with the builders right now. Once the issue is worked out, I'll tell the builders to retry your changes

[frewsxcv]

@bors-servo retry

• servo/saltfs#120

[bors-servo]

⌛ Testing commit 2e0e228 with merge 44de917...

[bors-servo]

☀️ Test successful - android, gonk, linux-dev, linux-rel, mac-dev-ref-unit, mac-rel-css, mac-rel-wpt