Pipreqs missing some requirements

[audiofeature]

I develop this project: https://github.com/tuwien-musicir/rp_extract

As you can see from the imports it uses a lot of libraries (numpy, scipy, matplotlib, argparse, unicsv, cPickle, sklearn, ...) - many of which are missed by pipreqs:

pipreqs --debug --force .
DEBUG: Found packages: set(['argparse', 'cPickle', 'uuid', 'tempfile', 'numpy', 'pylab', 'matplotlib', 'wave', 'sys', 'subprocess', 'os', 'sklearn'])
DEBUG: Found imports: matplotlib, numpy, matplotlib, scikit_learn
DEBUG: Getting packages information from Local/PyPI
INFO: Starting new HTTPS connection (1): pypi.python.org
DEBUG: Setting read timeout to None
DEBUG: "GET /pypi/numpy/json HTTP/1.1" 200 65657
INFO: Starting new HTTPS connection (1): pypi.python.org
DEBUG: Setting read timeout to None
DEBUG: "GET /pypi/scikit_learn/json HTTP/1.1" 301 0
DEBUG: Setting read timeout to None
DEBUG: "GET /pypi/scikit-learn/json HTTP/1.1" 200 102410
DEBUG: Writing 4 requirements: matplotlib, matplotlib, numpy, scikit_learn to ./requirements.txt
INFO: Successfully saved requirements file in ./requirements.txt
musimap@asus:~/rp_extract$ cat requirements.txt
matplotlib == 1.3.1.egg
matplotlib == 1.3.1.egg
numpy == 1.10.1
scikit_learn == 0.17

Why does it miss many of the library requirements?
(some of them are even mentioned in the debug output, some are not, e.g. scipy, unicsv)
(and matplotlib on the other hand is put twice into the requirements.txt)

[bndr]

Argparse and cpickle are part of standard libs of python, so they should't be in your requirements.txt
sklearn - is scikit_learn package.

Others are more interesting, how should the requirements.txt look in the end?

[audiofeature]

It should look like this (which I just succesfully tested):

numpy==1.8.2
scipy==0.13.3
argparse==1.2.1
unicsv==1.0.0
pandas==0.13.1
scikit-learn==0.17
matplotlib==1.3.1
pylab==0.1.3

More remarks:

1. If I use pipreqs, it puts numpy == 1.10.1 into the requirements.txt file because this is the latest version it gets via the URL from numpy. However, my own code is satisfied with numpy==1.8.2 (which is what I get when I use pip freeze | grep numpy ). Same with scikit-learn and others.
2. pipreqs clearly omits scipy, which is not even in the debug output, and pylab (which might be omitted because matplotlib requires it)
3. pip install -r requirements.txt fails when using matplotlib == 1.3.1.egg
   I had to remove .egg in order to have it work.

[bndr]

Thanks for reporting, I'll take a look at it.

[bndr]

What OS are you using? Python interpreter version?

[bndr]

To your first point: The version number is always the latest.

As to others: I just released version 0.3.4.
'.egg' should not be in version number anymore, scipy and other packages should appear correctly in your requirements file. Can you update and tell me if it works.

[audiofeature]

I updated to the latest pipreqs using
sudo pip install pipreqs==0.3.5

and now I get the usage printout everytime i call pipreqs, no matter what parameters I provide.

I use Python 2.7.10 on Ubuntu 15.10.

[audiofeature]

A comment on point #1 from the list above: I don't understand why you pull the latest version of all libraries from the web, while the version my own package uses works with another (lower) library version. In the end the requirements.txt will be incorrect, because if you need numpy==1.8.2 but pipreqs puts numpy == 1.10.1 into the requirements file there might be a an incompatibility with the new (future) version of numpy that nobody is aware of, which would be avoided when installing/requiring 1.8.2, the version that is proven to work with my program. (I use numpy as an arbitrary exemplary example here).

[bndr]

The requirements file is generated based on your imports. There is no way to know which version of the package you are using.

If your virtual environment has the used packages already installed it's better to use pip freeze, because then the list is generated based on the actual packages installed and not on your imports.

P.S 0.3.7 should fix your previous problem

[audiofeature]

Thanks for the update. The requirements.txt file generated now seems complete, except for argparse (which you say is part of the default python installation) and matplotlib still appearing twice.

It would be ideal to have a "mashup" between pip freeze and pipreqs. Can't you check for the installed version instead of the latest version available online for the libraries?

By this it could be guaranteed that the program works given the correct library dependencies + versions.

[bndr]

That is actually a good point. I'll see what I can do.
I'll close this issue seeing as the bug is fixed, and open another one for your suggestion of implementing some sort of mashup between imports and local packages.