Fix: pip support, especially pip10 (#657)

[screamerbg]

pip.get_installed_distributions() is no longer available in pip 10. This PR fixes this by using the user-facing commands of pip and ensure future and backwards compatibility.

Also bumps Mbed CLI version to 1.5.1

[alzix]

please note that -l is a deprecated option
it produces following warning:

DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.

consider using --format=legacy or --format=column instead

[screamerbg]

@alzix pip list -l is for listing the locally installed python modules. -l, --local If in a virtualenv that has global access, do not list globally-installed packages.

(pressed Ctrl+Enter by accident)

The problem with using --format=legacy is that it's not backwards compatible with pip < 10, so we can't use it either 😃

$ pip --version
pip 8.1.2 from /Library/Python/2.7/site-packages (python 2.7)

$ pip list -l --format=legacy

Usage:   
  pip list [options]

no such option: --format

[theotherjimmy]

I think we're going to either have to live with the deprecation warning, or add pip 10 as a dependency.

[screamerbg]

I've also tried forcing the columns format using the --format=columns option and the code above is handling it just fine:

[snip]
                pkg_list = pquery([python_cmd, '-m', 'pip', 'list', '-l', '--format=columns']) or ""
[snip]

$ pip uninstall pyyaml
Uninstalling PyYAML-3.12:
  Would remove:
    c:\progra~1\python\lib\site-packages\pyyaml-3.12-py2.7.egg-info
    c:\progra~1\python\lib\site-packages\yaml\*
Proceed (y/n)?
Your response ('') was not one of the expected responses: y, n
Proceed (y/n)? y
  Successfully uninstalled PyYAML-3.12

$ mbed update --clean
[mbed] Synchronizing dependency references...
[mbed] Updating program "mbed-os-example-wifi" to latest revision in the current branch
[mbed] Updating library "esp8266-driver" to rev #264468722a97
[mbed] Updating library "mbed-os" to rev #c05d72c3c005 (tag: mbed-os-5.8.3)
[mbed] Updating library "wifi-x-nucleo-idw01m1" to rev #5871f7011d7f
[mbed] Auto-installing missing Python modules...

$ pip list | grep -i pyyaml
PyYAML                        3.12

[theotherjimmy]

I think you're suggesting that we don't need to worry about the deprecation warning. That's how I took it at least.

[screamerbg]

Yes, exactly. Sorry that I didn't explicitly said it. Was writing this in between meetings 😀

[alzix]

i'm sorry for the confusion about the '-l/--local' but i vote for adding --format=columns to get rid of the deprecation warning

[screamerbg]

@alzix That would break the backwards compatibility with pip < 10. Where do you see the warning? When using mbed CLI or when using pip? I'm not seeing any warnings on Windows with pip 10.0.1

[screamerbg]

@alzix @theotherjimmy Note that this is now breaking the CI.

[alzix]

@screamerbg I see the deprecation warning on pip 9.0.3 Linux

[screamerbg]

@alzix I don't get any warnings from pip 9.0.3 on Mac when using mbed CLI.

$ sudo pip uninstall mbed-greentea
The directory '/Users/mihsto01/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Uninstalling mbed-greentea-1.3.3:
  /Library/Python/2.7/site-packages/mbed_greentea-1.3.3-py2.7.egg-info
  /Library/Python/2.7/site-packages/mbed_greentea/__init__.py
  /Library/Python/2.7/site-packages/mbed_greentea/__init__.pyc
[SNIP]
  /Library/Python/2.7/site-packages/test/report_api.py
  /Library/Python/2.7/site-packages/test/report_api.pyc
  /usr/local/bin/mbedgt
Proceed (y/n)? y
  Successfully uninstalled mbed-greentea-1.3.3
The directory '/Users/mihsto01/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
You are using pip version 9.0.3, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

$ sudo mbed update --clean
[mbed] Synchronizing dependency references...
[mbed] Updating program "mbed-os-example-wifi" to latest revision in the current branch
[mbed] Updating library "esp8266-driver" to rev #264468722a97
[mbed] Updating library "mbed-os" to rev #c05d72c3c005 (tags: latest, mbed-os-5.8.3)
[mbed] Updating library "wifi-x-nucleo-idw01m1" to rev #5871f7011d7f
[mbed] Auto-installing missing Python modules...

$ python --version
Python 2.7.10

$ pip --version
pip 9.0.3 from /Library/Python/2.7/site-packages (python 2.7)

I only get a warning when directly calling pip

$ pip list -l | grep -i greentea
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
mbed-greentea (1.3.3)
You are using pip version 9.0.3, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

[theotherjimmy]

@alzix I think @screamerbg is asking if you see the warning when using Mbed CLI from this branch. I don't see the warning on Linux with this branch and any version of Pip (I tried 9.0.3 and 10.0.1).

[cmonr]

LGTM!

[screamerbg]

Thanks. Just fixed the CI issue caused by faulty pip module;