AttributeError while installing package from PythonShell

[MarKett]

• Pip version: 9.0.1
• Python version: 3.6
• Operating system: Win7 - 64bit

Description:

• start a python 3.6 idle console
• import pip and try to install pylint
• at first that seems to work, than an AttributeError occurs while trying to install 'wrapt'

Cause:

In some cases sys.__stdout__ can be None, as noted here: https://docs.python.org/3/library/sys.html
pip does not check this in:
Python36\lib\site-packages\pip\compat\__init__.py
Line: 73
It tries to call sys.__stdout__.encoding and obviously fails

Solution:

Implement a check for sys.__stdout__ != None before trying to access it.

What I've run:

import pip
pip.main(["install", "pylint"])

StackTrace:

[...]
Collecting wrapt (from astroid>=1.5.1->pylint)
  Using cached wrapt-1.10.10.tar.gz
�[31mException:
Traceback (most recent call last):
  File "D:\Program Files\Python36\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "D:\Program Files\Python36\lib\site-packages\pip\commands\install.py", line 324, in run
    requirement_set.prepare_files(finder)
  File "D:\Program Files\Python36\lib\site-packages\pip\req\req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "D:\Program Files\Python36\lib\site-packages\pip\req\req_set.py", line 634, in _prepare_file
    abstract_dist.prep_for_dist()
  File "D:\Program Files\Python36\lib\site-packages\pip\req\req_set.py", line 129, in prep_for_dist
    self.req_to_install.run_egg_info()
  File "D:\Program Files\Python36\lib\site-packages\pip\req\req_install.py", line 439, in run_egg_info
    command_desc='python setup.py egg_info')
  File "D:\Program Files\Python36\lib\site-packages\pip\utils\__init__.py", line 676, in call_subprocess
    line = console_to_str(proc.stdout.readline())
  File "D:\Program Files\Python36\lib\site-packages\pip\compat\__init__.py", line 73, in console_to_str
    return s.decode(sys.__stdout__.encoding)
AttributeError: 'NoneType' object has no attribute 'encoding'�[0m

[pradyunsg]

Thanks for filing this issue!

Using pip by importing is not a supported way of using pip. The only supported way of using pip is to run it as a command line program.

That is, opening cmd.exe and using pip as follows:

C:\Windows\System32> pip install pylint

[pfmoore]

Also, this is likely fixed in the development version of pip. We changed this in #4486

[MarKett]

Using pip by importing is not a supported way of using pip. The only supported way of using pip is to run it as a command line program.

Two questions:
Whats the reason for that? It worked like a charm, when I set sys.__stdout__ = sys.stdout.
Shouldn't a program check for possible edge-cases that are documented in the sys modul documentation instead of just raising an exception?

I use pip in my python scripts to install packages on systems for our developers, so they don't have to handle this stuff for themselves. So the alternative would be to use os.subprocess(...) to call pip. That seems very dirty to me. Especially because it works in python, when catching this one exception.

[pfmoore]

Whats the reason for that?

Simply because the code wasn't written for that purpose, we never documented any guarantees for the internal APIs, and we don't have the time to support people using the internal API. Specifically, if we documented a supported API, we would have to ensure that any changes we made to the internals of pip maintained compatibility for the documented API.

So the alternative would be to use os.subprocess(...) to call pip. That seems very dirty to me.

No more so than calling git using os.subprocess. Pip is a command line program, the fact that it's implemented in Python is largely irrelevant to how you should use it.

Having said all of the above, realistically, using pip.main the way you do is probably OK (as long as you don't call it in the presence of threads, or care if it interacts oddly with your application's use of the logging package, etc... - both of those examples have been reported as actual issues BTW). But we don't offer support if things do go wrong. In this particular situation, though, the real answer is almost certainly "already fixed, will be available in the next version of pip".

[pradyunsg]

This would be a duplicate of #3356...

[MarKett]

That clarifies that.
Thank you for your time.

[pfmoore]

@MarKett In fact @pradyunsg pointed out the real issue. It's still not supported usage, but it's not hard to fix, so I've just pushed a fix for it so the problem should be resolved in the next version of pip.

Thanks for flagging the issue!