Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'pip list -v' not showing correct locations of installed modules #5861

Closed
legolego opened this issue Oct 6, 2018 · 27 comments
Closed

'pip list -v' not showing correct locations of installed modules #5861

legolego opened this issue Oct 6, 2018 · 27 comments
Labels
auto-locked Outdated issues that have been locked by automation type: support User Support

Comments

@legolego
Copy link

legolego commented Oct 6, 2018

Environment

  • pip version: 18
  • Python version: 3.7.1rc1 64 bit
  • OS: Windows 10
    Pycharm IDE

Description

I'm having trouble installing modules with pip and venv. I had one environment working fine, but in PyCharm I made a second one to be 64 bit. I'm trying to reinstall my modules again, but not succeeding. They seem to install successfully and show up in 'pip list -v' but they're not actually there. This has less to do with pip and maybe more with venv and my PATH, but 'pip list -v' isn't showing correct data. Also, maybe more of a request, I don't see the actual destination directory used to install a module when using the --verbose flag. It would be a big hint to know exactly where the module is being installed.

Expected behavior

I would expect 'pip install' and 'pip list' to use the same location variable, maybe they're using different locations to install to and show modules internally.

How to Reproduce

pip list -v

Output

pip list -v output doesn't match what's in the Lib\site-packages directory.

C:\gitProjects\lego\venv64\Lib\site-packages>dir
 Volume in drive C has no label.
 Volume Serial Number is 4A4B-BC58

 Directory of C:\gitProjects\lego\venv64\Lib\site-packages

10/05/2018  00:13    <DIR>          .
10/05/2018  00:13    <DIR>          ..
10/05/2018  00:13                53 easy-install.pth
10/05/2018  00:08    <DIR>          numpy
10/05/2018  00:08    <DIR>          numpy-1.15.2.dist-info
10/04/2018  23:58    <DIR>          pip-18.0-py3.7.egg
10/04/2018  23:56    <DIR>          pip-9.0.1-py3.7.egg
10/04/2018  23:56           465,038 setuptools-28.8.0-py3.7.egg
10/05/2018  00:13                31 setuptools.pth
               3 File(s)        465,122 bytes
               6 Dir(s)  53,285,056,512 bytes free
(venv64) C:\gitProjects\lego>pip list -v
Package         Version   Location                                                               Installer
--------------- --------- ---------------------------------------------------------------------- ---------
boto            2.49.0    c:\gitprojects\lego\venv64\lib\site-packages                    pip
boto3           1.9.16    c:\gitprojects\lego\venv64\lib\site-packages                    pip
botocore        1.12.16   c:\gitprojects\lego\venv64\lib\site-packages                    pip
bz2file         0.98      c:\gitprojects\lego\venv64\lib\site-packages
certifi         2018.8.24 c:\gitprojects\lego\venv64\lib\site-packages                    pip
chardet         3.0.4     c:\gitprojects\lego\venv64\lib\site-packages                    pip
corenlp         0.0.14    c:\gitprojects\lego\venv64\lib\site-packages
docutils        0.14      c:\gitprojects\lego\venv64\lib\site-packages                    pip
gensim          3.6.0     c:\gitprojects\lego\venv64\lib\site-packages
idna            2.7       c:\gitprojects\lego\venv64\lib\site-packages                    pip
jmespath        0.9.3     c:\gitprojects\lego\venv64\lib\site-packages                    pip
lxml            4.2.5     c:\gitprojects\lego\venv64\lib\site-packages                    pip
nltk            3.3       c:\gitprojects\lego\venv64\lib\site-packages
numpy           1.15.2    c:\gitprojects\lego\venv64\lib\site-packages                    pip
pandas          0.23.4    c:\gitprojects\lego\venv64\lib\site-packages                    pip
....
@benoit-pierre
Copy link
Member

So pip is saying that all those packages (boto, boto3, botocore, ...) are installed in c:\gitprojects\lego\venv64\lib\site-packages although they are not?

@legolego
Copy link
Author

legolego commented Oct 6, 2018

Yes, the 'dir' command above and 'pip list -v' were run at the same time.... there are many more packages shown with 'pip list -v', I'm pretty sure from my different venv.

@legolego
Copy link
Author

legolego commented Oct 6, 2018

This is my venv directory:

C:\gitProjects\lego\venv\Lib\site-packages>dir
 Volume in drive C has no label.
 Volume Serial Number is 4A4B-BC58

 Directory of C:\gitProjects\lego\venv\Lib\site-packages

10/02/2018  12:23    <DIR>          .
10/02/2018  12:23    <DIR>          ..
07/31/2018  19:49    <DIR>          backcall
07/31/2018  19:49    <DIR>          backcall-0.1.0.dist-info
07/31/2018  19:49    <DIR>          bleach
07/31/2018  19:49    <DIR>          bleach-2.1.3.dist-info
09/25/2018  13:23    <DIR>          blinker
09/25/2018  13:23    <DIR>          blinker-1.4.dist-info
10/02/2018  00:29    <DIR>          boto
10/02/2018  00:29    <DIR>          boto-2.49.0.dist-info
10/02/2018  00:29    <DIR>          boto3
10/02/2018  00:29    <DIR>          boto3-1.9.15.dist-info
10/02/2018  00:29    <DIR>          botocore
10/02/2018  00:29    <DIR>          botocore-1.12.15.dist-info
10/02/2018  00:29    <DIR>          bz2file-0.98.dist-info
10/02/2018  00:29            18,340 bz2file.py
08/06/2018  14:32    <DIR>          certifi
08/06/2018  14:32    <DIR>          certifi-2018.4.16.dist-info
08/06/2018  14:32    <DIR>          chardet
08/06/2018  14:32    <DIR>          chardet-3.0.4.dist-info
09/25/2018  13:19    <DIR>          click
09/25/2018  13:19    <DIR>          click-6.7.dist-info
07/31/2018  19:49    <DIR>          colorama
07/31/2018  19:49    <DIR>          colorama-0.3.9.dist-info
08/06/2018  14:32    <DIR>          corenlp
08/06/2018  14:32    <DIR>          corenlp_protobuf
08/06/2018  14:32    <DIR>          corenlp_protobuf-3.8.0.dist-info
07/31/2018  19:49    <DIR>          cycler-0.10.0.dist-info
07/31/2018  19:47            15,959 cycler.py
10/02/2018  11:31    <DIR>          Cython
10/02/2018  11:31    <DIR>          Cython-0.28.5.dist-info
10/02/2018  11:30               520 cython.py
07/31/2018  19:49    <DIR>          dateutil
07/31/2018  19:49    <DIR>          decorator-4.3.0.dist-info
07/31/2018  19:47            16,298 decorator.py
10/02/2018  00:29    <DIR>          docutils
10/02/2018  00:29    <DIR>          docutils-0.14.dist-info
07/31/2018  17:57                 0 easy-install.pth
07/31/2018  17:57               126 easy_install.py
07/31/2018  19:49    <DIR>          entrypoints-0.2.3.dist-info
07/31/2018  19:48             6,997 entrypoints.py
10/02/2018  11:42    <DIR>          fastText
10/02/2018  11:42    <DIR>          fasttext-0.8.22.dist-info
10/02/2018  11:42           318,464 fasttext_pybind.cp37-win32.pyd
09/25/2018  13:19    <DIR>          flask
09/25/2018  13:19    <DIR>          Flask-1.0.2.dist-info
09/25/2018  13:23    <DIR>          Flask_Mail-0.9.1.dist-info
09/25/2018  13:23            17,950 flask_mail.py
10/02/2018  12:23    <DIR>          future
10/02/2018  12:23    <DIR>          future-0.16.0.dist-info
10/02/2018  00:29    <DIR>          gensim
10/02/2018  00:29    <DIR>          gensim-3.6.0.dist-info
08/06/2018  14:32    <DIR>          google
07/31/2018  19:49    <DIR>          html5lib
07/31/2018  19:49    <DIR>          html5lib-1.0.1.dist-info
08/06/2018  14:32    <DIR>          idna
08/06/2018  14:32    <DIR>          idna-2.7.dist-info
07/31/2018  19:49    <DIR>          ipykernel
07/31/2018  19:49    <DIR>          ipykernel-4.8.2.dist-info
07/31/2018  19:48               451 ipykernel_launcher.py
07/31/2018  19:49    <DIR>          IPython
07/31/2018  19:49    <DIR>          ipython-6.5.0.dist-info
07/31/2018  19:49    <DIR>          ipython_genutils
07/31/2018  19:49    <DIR>          ipython_genutils-0.2.0.dist-info
07/31/2018  19:50    <DIR>          ipywidgets
07/31/2018  19:50    <DIR>          ipywidgets-7.3.1.dist-info
07/31/2018  19:48            11,199 isympy.py
09/25/2018  13:19    <DIR>          itsdangerous-0.24.dist-info
09/25/2018  13:19            31,840 itsdangerous.py
07/31/2018  19:49    <DIR>          jedi
07/31/2018  19:49    <DIR>          jedi-0.12.1.dist-info
07/31/2018  19:49    <DIR>          jinja2
07/31/2018  19:49    <DIR>          Jinja2-2.10.dist-info
10/02/2018  00:29    <DIR>          jmespath
10/02/2018  00:29    <DIR>          jmespath-0.9.3.dist-info
07/31/2018  19:49    <DIR>          jsonschema
07/31/2018  19:49    <DIR>          jsonschema-2.6.0.dist-info
07/31/2018  19:50    <DIR>          jupyter-1.0.0.dist-info
07/31/2018  19:47               177 jupyter.py
07/31/2018  19:49    <DIR>          jupyter_client
07/31/2018  19:49    <DIR>          jupyter_client-5.2.3.dist-info
07/31/2018  19:50    <DIR>          jupyter_console
07/31/2018  19:50    <DIR>          jupyter_console-5.2.0.dist-info
07/31/2018  19:49    <DIR>          jupyter_core
07/31/2018  19:49    <DIR>          jupyter_core-4.4.0.dist-info
07/31/2018  19:49    <DIR>          kiwisolver-1.0.1.dist-info
07/31/2018  19:47           108,544 kiwisolver.cp37-win32.pyd
10/02/2018  12:23    <DIR>          libfuturize
10/02/2018  12:23    <DIR>          libpasteurize
07/31/2018  18:03    <DIR>          lxml
07/31/2018  18:03    <DIR>          lxml-4.2.3.dist-info
07/31/2018  19:49    <DIR>          markupsafe
07/31/2018  19:49    <DIR>          MarkupSafe-1.0.dist-info
07/31/2018  19:49    <DIR>          matplotlib
07/31/2018  19:47               570 matplotlib-2.2.2-py3.7-nspkg.pth
07/31/2018  19:49    <DIR>          matplotlib-2.2.2.dist-info
07/31/2018  19:49    <DIR>          mistune-0.8.3.dist-info
07/31/2018  19:48            35,484 mistune.py
07/31/2018  19:49    <DIR>          mpl_toolkits
07/31/2018  19:51    <DIR>          mpmath
07/31/2018  19:51    <DIR>          mpmath-1.0.0.dist-info
07/31/2018  19:49    <DIR>          nbconvert
07/31/2018  19:49    <DIR>          nbconvert-5.3.1.dist-info
07/31/2018  19:49    <DIR>          nbformat
07/31/2018  19:49    <DIR>          nbformat-4.4.0.dist-info
08/27/2018  22:07    <DIR>          neo4j
08/27/2018  22:07    <DIR>          neo4j-1.7.0b2.dist-info
07/31/2018  14:29    <DIR>          neo4j_driver-1.6.1-py3.7.egg-info
08/27/2018  22:07    <DIR>          neobolt
08/27/2018  22:07    <DIR>          neobolt-1.7.0b2.dist-info
07/31/2018  14:29    <DIR>          neotime
07/31/2018  14:29    <DIR>          neotime-1.0.0-py3.7.egg-info
08/07/2018  15:05    <DIR>          nltk
08/07/2018  15:05    <DIR>          nltk-3.3.dist-info
07/31/2018  19:52    <DIR>          nose
07/31/2018  19:52    <DIR>          nose-1.3.7.dist-info
07/31/2018  19:50    <DIR>          notebook
07/31/2018  19:50    <DIR>          notebook-5.6.0.dist-info
07/31/2018  17:53    <DIR>          numpy
07/31/2018  17:53    <DIR>          numpy-1.15.0+mkl.dist-info
07/31/2018  19:51    <DIR>          pandas
07/31/2018  19:51    <DIR>          pandas-0.23.3.dist-info
07/31/2018  19:49    <DIR>          pandocfilters-1.4.2.dist-info
....

@benoit-pierre
Copy link
Member

Did you rename the venv, or move it's location?

@legolego
Copy link
Author

legolego commented Oct 6, 2018

'pip install' seems to work, but it's not in the current venv64 directory, and in the venv directory there aren't any new timestamps, so it's not installed there either.

(venv64) C:\gitProjects\lego>pip install --upgrade --force-reinstall --ignore-installed --no-cache-dir corenlp
Collecting corenlp
  Downloading https://files.pythonhosted.org/packages/83/b7/c6218e87078a25bfb48bbb8e8e9055e83cb8b49b7c58481ce4cebf0b5227/corenlp-0.0.14.tar.gz
Collecting nltk (from corenlp)
  Downloading https://files.pythonhosted.org/packages/50/09/3b1755d528ad9156ee7243d52aa5cd2b809ef053a0f31b53d92853dd653a/nltk-3.3.0.zip (1.4MB)
    100% |████████████████████████████████| 1.4MB 8.6MB/s
Collecting six (from nltk->corenlp)
  Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Installing collected packages: six, nltk, corenlp
  Running setup.py install for nltk ... done
  Running setup.py install for corenlp ... done
Successfully installed corenlp-0.0.14 nltk-3.3 six-1.11.0


@benoit-pierre
Copy link
Member

Don't use --ignore-installed, never a good idea...

@legolego
Copy link
Author

legolego commented Oct 6, 2018

No, I didn't move the venv directory... I tried making a second venv (venv64 with 64 bit Python) but I can't install any modules into it.

@benoit-pierre
Copy link
Member

Why can't you install anything? Are you sure you're using the right pip? The one from the venv, and not one from your global Python installation? Check the output of pip --version, this will show the location where pip is installed.

@legolego
Copy link
Author

legolego commented Oct 6, 2018

yes, it seems to be....


(venv64) C:\gitProjects\lego>pip --version
pip 18.0 from c:\gitprojects\lego\venv64\lib\site-packages\pip-18.0-py3.7.egg\pip (python 3.7)

@benoit-pierre
Copy link
Member

OK, so after installing a package, what's the issue with using it?

@benoit-pierre
Copy link
Member

Note the fact that pip is installed as an egg is weird... That's not how it normally works. I know PyCharm does install packages as eggs, and this is known to cause issues.

@legolego
Copy link
Author

legolego commented Oct 6, 2018

It's not showing in 'pip list -v' like that corenlp install for example... it installs 'successfully' but it's not really there. I'm sure I'm doing something wrong, but wondering if this is a bug in pip too.

@benoit-pierre
Copy link
Member

It's going to be difficult to debug further without the exact steps: starting with the command used to create the venv, how you activate it, ... Are you doing all this from PyCharm?

@legolego
Copy link
Author

legolego commented Oct 6, 2018

Yes, I went into project settings to create it and point to a the 64 bit Python interpreter.

Is there a way to see which directory in the 'install' command is using? Maybe with verbose?
I don't understand what is meant by: 'Option is additive, and can be used up to 3 times.'

@benoit-pierre
Copy link
Member

The more -v you pass, the more verbose the output is.

@legolego
Copy link
Author

legolego commented Oct 6, 2018

Ok, I tried that with an install, and minus the --ignore-installed flag

pip install --upgrade --force-reinstall --no-cache-dir -v -v -v corenlp

Getting some lines like this(this is what I wanted to see in the verbose output):

running install_lib
    creating C:\gitProjects\lego\venv64\Lib\site-packages\nltk
    creating C:\gitProjects\lego\venv64\Lib\site-packages\nltk\app
    copying build\lib\nltk\app\chartparser_app.py -> C:\gitProjects\lego\venv64\Lib\site-packages\nltk\app
    copying build\lib\nltk\app\chunkparser_app.py -> C:\gitProjects\lego\venv64\Lib\site-packages\nltk\app
    copying build\lib\nltk\app\collocations_app.py -> C:\gitProjects\lego\venv64\Lib\site-packages\nltk\app

so an nltk directory looks like it's being created, but it's not really there.

Later it removes corenlp that was never there:

Found existing installation: corenlp 0.0.14
    Uninstalling corenlp-0.0.14:
      Created temporary directory: C:\Users\Oleg\AppData\Local\Temp\pip-uninstall-4u3xjuz1
      Removing file or directory c:\gitprojects\lego\venv64\lib\site-packages\corenlp-0.0.14-py3.7.egg-info
      Removing file or directory c:\gitprojects\lego\venv64\lib\site-packages\corenlp\__init__.py
      Removing file or directory c:\gitprojects\lego\venv64\lib\site-packages\corenlp\__pycache__\__init__.cpython-37.pyc

but then here fails (where it usually succeeds), but there's still nothing new in \venv64\Lib\site-packages

Successfully installed corenlp-0.0.14 nltk-3.3 six-1.11.0
Cleaning up...
Removed build tracker 'C:\\Users\\Oleg\\AppData\\Local\\Temp\\pip-req-tracker-xlzgwnym'
There was an error checking the latest version of pip
Traceback (most recent call last):
  File "c:\gitprojects\lego\venv64\lib\site-packages\pip-18.0-py3.7.egg\pip\_internal\utils\outdated.py", line 90, in pip_version_check
    state = SelfCheckState(cache_dir=options.cache_dir)
  File "c:\gitprojects\lego\venv64\lib\site-packages\pip-18.0-py3.7.egg\pip\_internal\utils\outdated.py", line 25, in __init__
    self.statefile_path = os.path.join(cache_dir, "selfcheck.json")
  File "C:\Program Files\Python37\lib\ntpath.py", line 76, in join
    path = os.fspath(path)
TypeError: expected str, bytes or os.PathLike object, not bool

@benoit-pierre
Copy link
Member

That last error should be fixed in 18.1.

@legolego
Copy link
Author

legolego commented Oct 6, 2018

What's the right way to switch the location of pip that is being used? I started a new Project in PyCharm, and made a new venv, but the old version of pip is being used still.

@benoit-pierre
Copy link
Member

I'm sorry, but I don't use PyCharm. Maybe try invoking pip using python -m pip instead? You also might have more luck asking on PyCharm issue tracker.

@legolego
Copy link
Author

legolego commented Oct 6, 2018

I tried that and 'python -m pip list -v' shows correctly

but I get this error trying an install:

(venv) C:\gitProjects\lego3>python -m pip install --upgrade --no-cache-dir corenlp
Collecting corenlp
  Downloading https://files.pythonhosted.org/packages/83/b7/c6218e87078a25bfb48bbb8e8e9055e83cb8b49b7c58481ce4cebf0b5227/corenlp-0.0.14.tar.gz
    Error [WinError 87] The parameter is incorrect while executing command python setup.py egg_info
Could not install packages due to an EnvironmentError: [WinError 87] The parameter is incorrect

Would that be related to my PATH at all?

@benoit-pierre
Copy link
Member

No idea, use -v to get more information.

@benoit-pierre
Copy link
Member

Latest corenlp release is from 2014... Forget Python 3 support.

@legolego
Copy link
Author

legolego commented Oct 6, 2018

This seems to be working, thank you very much! :)

python -m pip install -v stanford-corenlp

@benoit-pierre
Copy link
Member

👍

@pradyunsg pradyunsg added the type: support User Support label Oct 8, 2018
@pradyunsg
Copy link
Member

Thanks @benoit-pierre for helping out @legolego! :)

@pradyunsg
Copy link
Member

@legolego It seems your issue is resolved, so I'll go ahead and close this issue.

You can still comment on the closed to let us know if that is not the case -- I'll reopen this issue in that case.

@lock
Copy link

lock bot commented May 31, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot added the auto-locked Outdated issues that have been locked by automation label May 31, 2019
@lock lock bot locked as resolved and limited conversation to collaborators May 31, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
auto-locked Outdated issues that have been locked by automation type: support User Support
Projects
None yet
Development

No branches or pull requests

3 participants