Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Release and Hypervolume warning #240

Closed
DMTSource opened this issue Nov 12, 2017 · 34 comments
Closed

New Release and Hypervolume warning #240

DMTSource opened this issue Nov 12, 2017 · 34 comments

Comments

@DMTSource
Copy link

My Ubuntu 14.04, after upgrading to the new pip release(omg thank you by the way!), has begun throwing the Hypervolume import warning. I am used to seeing this harmless(?) warning in my windows setup using older pip but I don't believe I was seeing it before in Linux when I had installed directly from the master via:
pip install git+git://github.com/DEAP/deap/

Is this expected behavior for the pip release?

/usr/local/lib/python2.7/dist-packages/deap/tools/_hypervolume/pyhv.py:33: ImportWarning: Falling back to the python version of hypervolume module. Expect this to be very slow.
  "module. Expect this to be very slow.", ImportWarning)
@fmder
Copy link
Member

fmder commented Nov 12, 2017

You are right, it looks like it does not get installed. I've update the release. 1.2.1 should fix this. Two release the same day! It will probably take another 4 years before the next one ;)

@fmder fmder closed this as completed Nov 12, 2017
@fmder fmder reopened this Nov 12, 2017
@fmder
Copy link
Member

fmder commented Nov 12, 2017

Seems like pip install didn't work... I'm working on that

@fmder fmder closed this as completed Nov 12, 2017
@fmder fmder reopened this Nov 12, 2017
@fmder
Copy link
Member

fmder commented Nov 12, 2017

Finally I got it working 1.2.2 is out now.

@fmder fmder closed this as completed Nov 12, 2017
@weixuanfu
Copy link

weixuanfu commented Nov 14, 2017

I still got this ImportError message with deap1.2.2

OS: Windows 10
Python 3.6.3

Python 3.6.3 |Anaconda custom (64-bit)| (default, Oct 15 2017, 03:27:45) [MSC v.1900 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 6.1.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from deap import gp
D:\Anaconda3\lib\site-packages\deap\tools\_hypervolume\pyhv.py:33: ImportWarning: Falling back to the python version of hypervolume module. Expect this to be very slow.
  "module. Expect this to be very slow.", ImportWarning)

@DMTSource
Copy link
Author

Just want to add my observations as well:
After the new 1.2.2 fix my Ubuntu environment with 2.7 was no longer showing the Hypervolume as expected.
But, it is still showing the warning on Windows 7 with python 2.7, but that was always the case for me so I assumed things were back to working normally.

@fmder
Copy link
Member

fmder commented Nov 14, 2017

You must compile the extension. I'm not sure how to do it on Windows. That is the difference between the Ubuntu and Windows use.

@weixuanfu
Copy link

It can be reproduced in MacOS too. Please check the log below:

[Tue Nov 14 14:13:39 weixuanfu@vagelos-ve505-0877:~ ] $ pip install deap --upgrade
Collecting deap
  Using cached deap-1.2.2.tar.gz
Building wheels for collected packages: deap
  Running setup.py bdist_wheel for deap ... done
  Stored in directory: /Users/weixuanfu/Library/Caches/pip/wheels/82/aa/67/2c93e17c84646c86099fda53ee0b3329372dcf94dd8789fd13
Successfully built deap
Installing collected packages: deap
  Found existing installation: deap 1.0.2
    Uninstalling deap-1.0.2:
      Successfully uninstalled deap-1.0.2
Successfully installed deap-1.2.2
[Tue Nov 14 14:14:07 weixuanfu@vagelos-ve505-0877:~ ] $ ipython
Python 3.6.0 |Anaconda custom (x86_64)| (default, Dec 23 2016, 13:19:00) 
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: from deap import gp
/Users/weixuanfu/anaconda3/lib/python3.6/site-packages/deap/tools/_hypervolume/pyhv.py:33: ImportWarning: Falling back to the python version of hypervolume module. Expect this to be very slow.
  "module. Expect this to be very slow.", ImportWarning)

@fmder
Copy link
Member

fmder commented Nov 14, 2017

Have your tried to delete everything related to deap in your site-packages? Can you run pip install in verbose mode?

@weixuanfu
Copy link

weixuanfu commented Nov 14, 2017

@fmder @hossein-pourbozorg Thank you for the solutions. I tried both but the warning message still shows in my MacOS (10.13). I even create a new conda environment as the log below. The warning message still raised.

[Tue Nov 14 15:03:20 weixuanfu@vagelos-ve505-0877:~ ] $ conda create -n testenv --yes python=3.6
Fetching package metadata ...........
Solving package specifications: .

Package plan for installation in environment /Users/weixuanfu/anaconda3/envs/testenv:

The following NEW packages will be INSTALLED:

    certifi:    2016.2.28-py36_0
    openssl:    1.0.2l-0        
    pip:        9.0.1-py36_1    
    python:     3.6.2-0         
    readline:   6.2-2           
    setuptools: 36.4.0-py36_1   
    sqlite:     3.13.0-0        
    tk:         8.5.18-0        
    wheel:      0.29.0-py36_0   
    xz:         5.2.3-0         
    zlib:       1.2.11-0        

openssl-1.0.2l 100% |################################################################################################################| Time: 0:00:00   6.78 MB/s
xz-5.2.3-0.tar 100% |################################################################################################################| Time: 0:00:00  18.31 MB/s
zlib-1.2.11-0. 100% |################################################################################################################| Time: 0:00:00  21.72 MB/s
python-3.6.2-0 100% |################################################################################################################| Time: 0:00:01  11.29 MB/s
certifi-2016.2 100% |################################################################################################################| Time: 0:00:00   9.24 MB/s
wheel-0.29.0-p 100% |################################################################################################################| Time: 0:00:00  12.04 MB/s
setuptools-36. 100% |################################################################################################################| Time: 0:00:00  17.50 MB/s
pip-9.0.1-py36 100% |################################################################################################################| Time: 0:00:00  24.13 MB/s
#
# To activate this environment, use:
# > source activate testenv
#
# To deactivate an active environment, use:
# > source deactivate
#

[Tue Nov 14 15:04:33 weixuanfu@vagelos-ve505-0877:~ ] $ source activate testenv
(testenv) [Tue Nov 14 15:05:34 weixuanfu@vagelos-ve505-0877:~ ] $ pip install deap
Collecting deap
Installing collected packages: deap
Successfully installed deap-1.2.2
(testenv) [Tue Nov 14 15:05:43 weixuanfu@vagelos-ve505-0877:~ ] $ python
Python 3.6.2 |Continuum Analytics, Inc.| (default, Jul 20 2017, 13:14:59) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from deap import gp
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/weixuanfu/anaconda3/envs/testenv/lib/python3.6/site-packages/deap/gp.py", line 34, in <module>
    from . import tools        # Needed by HARM-GP
  File "/Users/weixuanfu/anaconda3/envs/testenv/lib/python3.6/site-packages/deap/tools/__init__.py", line 26, in <module>
    from .indicator import *
  File "/Users/weixuanfu/anaconda3/envs/testenv/lib/python3.6/site-packages/deap/tools/indicator.py", line 17, in <module>
    import numpy
ModuleNotFoundError: No module named 'numpy'
>>> exit()
(testenv) [Tue Nov 14 15:06:06 weixuanfu@vagelos-ve505-0877:~ ] $ conda install numpy scipy
Fetching package metadata ...........
Solving package specifications: .

Package plan for installation in environment /Users/weixuanfu/anaconda3/envs/testenv:

The following NEW packages will be INSTALLED:

    mkl:   2017.0.3-0        
    numpy: 1.13.1-py36_0     
    scipy: 0.19.1-np113py36_0

Proceed ([y]/n)? y

numpy-1.13.1-p 100% |################################################################################################################| Time: 0:00:00   9.70 MB/s
scipy-0.19.1-n 100% |################################################################################################################| Time: 0:00:00  21.86 MB/s
(testenv) [Tue Nov 14 15:06:44 weixuanfu@vagelos-ve505-0877:~ ] $ python
Python 3.6.2 |Continuum Analytics, Inc.| (default, Jul 20 2017, 13:14:59) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from deap import gp
/Users/weixuanfu/anaconda3/envs/testenv/lib/python3.6/site-packages/deap/tools/_hypervolume/pyhv.py:33: ImportWarning: Falling back to the python version of hypervolume module. Expect this to be very slow.
  "module. Expect this to be very slow.", ImportWarning)

@fmder
Copy link
Member

fmder commented Nov 14, 2017

Can you do

pip install deap --verbose?

@weixuanfu
Copy link

weixuanfu commented Nov 14, 2017

Below is the log of pip install deap --verbose:

(testenv) [Tue Nov 14 15:35:57 weixuanfu@vagelos-ve505-0877:~ ] $ pip install deap --verbose
Collecting deap
  1 location(s) to search for versions of deap:
  * https://pypi.python.org/simple/deap/
  Getting page https://pypi.python.org/simple/deap/
  Looking up "https://pypi.python.org/simple/deap/" in the cache
  Current age based on date: 2319
  Freshness lifetime from max-age: 600
  Freshness lifetime from request max-age: 600
  The cached response is "stale" with no etag, purging
  Starting new HTTPS connection (1): pypi.python.org
  "GET /simple/deap/ HTTP/1.1" 200 1144
  Updating cache with response from "https://pypi.python.org/simple/deap/"
  Caching b/c date exists and max-age > 0
  Analyzing links from page https://pypi.python.org/simple/deap/
    Found link https://pypi.python.org/packages/16/06/d0b4b9afcdeb32005ae2f2a4d8d8b5e310f8549193b6aff720a9f6422bbc/deap-1.2.1a0.tar.gz#md5=ce4ae25e94cc109e46e538cdd087b337 (from https://pypi.python.org/simple/deap/), version: 1.2.1a0
    Found link https://pypi.python.org/packages/26/d1/d60043abbd3c8c9b856a83bcefaff0095fe4b4e5e4a5eab47fd7bea1419f/deap-1.2.1a1.tar.gz#md5=483e1168bfcbea482b1e16a06651afd2 (from https://pypi.python.org/simple/deap/), version: 1.2.1a1
    Found link https://pypi.python.org/packages/5b/d7/a49d3dd7aa8cbaf2b1ac8f4d6495824c886fea8b3dac4a73dc4df94cad76/deap-1.0.2.post2.tar.gz#md5=ccf5ed7562e4d6236c9416e3b5a9d941 (from https://pypi.python.org/simple/deap/), version: 1.0.2.post2
    Found link https://pypi.python.org/packages/70/25/a58a5eee102f9486b9aa729e43de438c6cedbe6dbad8a1705a2902311397/deap-1.0.1.tar.gz#md5=f1d948500ac658af6d072cb2e74030d9 (from https://pypi.python.org/simple/deap/), version: 1.0.1
    Found link https://pypi.python.org/packages/7a/00/a67c0751b8bda3b419ec942a7cc532bf4bda666b4cfe27eb355e53159190/deap-1.0.0.tar.gz#md5=59a5aee238326e6d948883a6b2b12924 (from https://pypi.python.org/simple/deap/), version: 1.0.0
    Found link https://pypi.python.org/packages/8c/f8/693ae0704656b6c0f44cc6e693b058dbaaf0af0d28e5ee1332152ff5fa49/deap-0.9.1.tar.gz#md5=d844135eb93573c45e45f6dd8b15e2dc (from https://pypi.python.org/simple/deap/), version: 0.9.1
    Found link https://pypi.python.org/packages/90/c6/946f7474b18600ae189fc5b81fa3ceac6a7bed7633d0825a8e1cb16f72ea/deap-1.0.0rc3.tar.gz#md5=099279fd19578ca32f50ecf46988d87b (from https://pypi.python.org/simple/deap/), version: 1.0.0rc3
    Found link https://pypi.python.org/packages/af/29/e7f2ecbe02997b16a768baed076f5fc4781d7057cd5d9adf7c94027845ba/deap-1.2.2.tar.gz#md5=3d2cdcea1cd38f61696f93f5aaee9b4e (from https://pypi.python.org/simple/deap/), version: 1.2.2
    Found link https://pypi.python.org/packages/b0/9c/a7404777a4cdf5857224051f2ae363d76c3d3b17d27b2caf12ed9bbe6d94/deap-1.2.1.tar.gz#md5=80807760213eb696f0263e214d11fb04 (from https://pypi.python.org/simple/deap/), version: 1.2.1
    Found link https://pypi.python.org/packages/b5/63/ea195522c9e987d0b497bbbd8c1c7183f6c39c886637a6eaa8c6b33519f7/deap-1.2.0.tar.gz#md5=afd5c8c9b30c2c089f799d16485330ec (from https://pypi.python.org/simple/deap/), version: 1.2.0
    Found link https://pypi.python.org/packages/d5/ca/2cf98e1e0d4e3643d5ac2097136bfd6a401414a324629655d23a1fab26fd/deap-0.9.2.tar.gz#md5=9455e43a88f9f3afc46163f499c8eae5 (from https://pypi.python.org/simple/deap/), version: 0.9.2
    Found link https://pypi.python.org/packages/dd/6f/4611020289cf6961e969720c975b9c3a6c06b8ac7e64877269789790a9b1/deap-1.2.1rc3.tar.gz#md5=5e51290527c14a1bce67ef0c86475a51 (from https://pypi.python.org/simple/deap/), version: 1.2.1rc3
    Found link https://pypi.python.org/packages/f9/1a/9edd5d06d132d29e47d47607a4c623e3dea4320aa638b45510badca02397/deap-1.2.1b0.tar.gz#md5=77953ad7c713c5cdedc46a4007e357b5 (from https://pypi.python.org/simple/deap/), version: 1.2.1b0
    Found link https://pypi.python.org/packages/f9/bc/48480f254ffd48003ba2eb787fca8003a180edc010b05a28aa680b982f38/deap-1.2.1a2.tar.gz#md5=3c8f957574ad13fb23d4efa3917376b7 (from https://pypi.python.org/simple/deap/), version: 1.2.1a2
  Using version 1.2.2 (newest of versions: 0.9.1, 0.9.2, 1.0.0, 1.0.1, 1.0.2.post2, 1.2.0, 1.2.1, 1.2.2)
  Using cached wheel link: file:///Users/weixuanfu/Library/Caches/pip/wheels/82/aa/67/2c93e17c84646c86099fda53ee0b3329372dcf94dd8789fd13/deap-1.2.2-cp36-none-any.whl
Installing collected packages: deap

Successfully installed deap-1.2.2
Cleaning up...

@fmder
Copy link
Member

fmder commented Nov 14, 2017

It seems like it built a wheel, use --no-cache-dir

what is the output of:

gcc --version?

@weixuanfu
Copy link

@fmder Thank you for the quick reply, but it still does not work with --no-cache-dir
gcc version = 4.8.5

Stdout Log:
install_deap_log_1711140354.txt

@fmder
Copy link
Member

fmder commented Nov 14, 2017

It seems that you don't have stdlib.h. That is really weird. How did you installed Python?

gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/weixuanfu/anaconda3/envs/testenv/include -arch x86_64 -I/Users/weixuanfu/anaconda3/envs/testenv/include -arch x86_64 -I/Users/weixuanfu/anaconda3/envs/testenv/include/python3.6m -c deap/tools/_hypervolume/_hv.c -o build/temp.macosx-10.7-x86_64-3.6/deap/tools/_hypervolume/_hv.o
deap/tools/_hypervolume/_hv.c:52:20: fatal error: stdlib.h: No such file or directory
#include <stdlib.h>
^
compilation terminated.

@weixuanfu
Copy link

I installed the python environment via the conda command below:

conda create -n testenv --yes python=3.6

(testenv) [Tue Nov 14 15:59:41 weixuanfu@vagelos-ve505-0877:~ ] $ conda list
# packages in environment at /Users/weixuanfu/anaconda3/envs/testenv:
#
certifi                   2016.2.28                py36_0  
deap                      1.2.2                     <pip>
mkl                       2017.0.3                      0  
numpy                     1.13.1                   py36_0  
openssl                   1.0.2l                        0  
pip                       9.0.1                    py36_1  
python                    3.6.2                         0  
readline                  6.2                           2  
scipy                     0.19.1              np113py36_0  
setuptools                36.4.0                   py36_1  
sqlite                    3.13.0                        0  
tk                        8.5.18                        0  
wheel                     0.29.0                   py36_0  
xz                        5.2.3                         0  
zlib                      1.2.11                        0  

@fmder
Copy link
Member

fmder commented Nov 14, 2017

Do you have xcode installed?

@weixuanfu
Copy link

weixuanfu commented Nov 14, 2017

Yep

(testenv) [Tue Nov 14 16:00:09 weixuanfu@vagelos-ve505-0877:~ ] $ /usr/bin/xcodebuild -version
Xcode 8.3.3
Build version 8E3004b

@fmder
Copy link
Member

fmder commented Nov 15, 2017

Something is wrong with your compiling system. stdlib.h is pretty standard header. I cannot help you any further.

@fmder
Copy link
Member

fmder commented Nov 15, 2017

Maybe you can try

xcode-select --install

from here

@weixuanfu
Copy link

weixuanfu commented Nov 15, 2017

OK, I figured out what happened about gcc compiler. The test environment used gcc (v.4.8.5) in Anaconda but not system default gcc (v.4.2.1). I changed the system environment to make pip to use system default gcc to re-compile deap. Then no warning message after importing deap. Thanks for your help.

@fmder
Copy link
Member

fmder commented Nov 15, 2017

Great!

@danelee2601
Copy link

Hey, guys, I had the exact same problem and error message.
I installed and uninstalled and reinstalled it again and again but it failed.
The problem was that I installed it with anaconda prompt with "conda install deap" <- it was the problem.
[I'm using python 3.6 with anaconda installation , window7]

The problem solved when I installed it as follows.
conda install -c conda-forge deap (ref : https://anaconda.org/conda-forge/deap)

and the error message no longer showed up.

image

@waelajaber
Copy link

thank you @danelee2601 it worked for me.

@tolgacesur
Copy link

@hossein-pourbozorg thanks. it worked!

@ZengLi0910
Copy link

I have the same problem and same warning massage when I work with python3.6, MacOS. but when I change it to python2.7 and reinstall it using @danelee2601's method, the massage disappeared. I'm wondering whether it will cause some trouble to my results... and thank you @danelee2601 ~
thanks~

@fmder
Copy link
Member

fmder commented Mar 16, 2018 via email

@ZengLi0910
Copy link

ZengLi0910 commented Mar 17, 2018 via email

@kramer65
Copy link

kramer65 commented Apr 13, 2018

I'm having the same problem on Ubuntu 18.04 (I know 18.04 is not released yet, but only 2 weeks until release and already very stable). I first installed using the usual sudo pip install deap, which succesfully installed version 1.2.2, but importing it gives the same warning:

$ python
Python 2.7.14+ (default, Apr  2 2018, 04:16:25) 
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import deap
>>> from deap import algorithms
/usr/local/lib/python2.7/dist-packages/deap/tools/_hypervolume/pyhv.py:33: ImportWarning: Falling back to the python version of hypervolume module. Expect this to be very slow.
  "module. Expect this to be very slow.", ImportWarning)
>>> 

I then tried installing it in a virtualenv, with the same result.

My next try was the suggestion by hossein-pourbozorg above both system wide and in a virtualenv:

pip install --upgrade --upgrade-strategy "eager" --force-reinstall --ignore-installed --compile --process-dependency-links --no-binary :all: deap

But unfortunately I still get the same error.

Does anybody know how I can solve this?

@fmder
Copy link
Member

fmder commented Apr 13, 2018

Have you installed the python-dev and build-essentials?

@kramer65
Copy link

@fmder - I didn't have python-dev yet. Installed it and reinstalled deap using the command below, which solved the issue.

pip install --upgrade --upgrade-strategy "eager" --force-reinstall --ignore-installed --compile --process-dependency-links --no-binary :all: deap

Thanks a million. Also for creating deap!

@850075610
Copy link

pip install --upgrade --upgrade-strategy "eager" --force-reinstall --ignore-installed --compile --process-dependency-links --no-binary :all: deap

This solution also works in win 10. Thanks very much!

@jarlva
Copy link

jarlva commented Oct 27, 2018

The solution above did not work. Using Windows 10/Python 3.6.6/pip (not conda).

pip install --upgrade --upgrade-strategy "eager" --force-reinstall --ignore-installed --compile --process-dependency-links --no-binary :all: deap

Error:
c:\users\userx\py\36env\lib\site-packages\deap\tools_hypervolume\pyhv.py:33: ImportWarning: Falling back to the python version of hypervolume module. Expect this to be very slow.
"module. Expect this to be very slow.", ImportWarning)
C:\Users\userx\AppData\Local\Programs\Python\Python36\lib\importlib_bootstrap.py:219: ImportWarning: can't resolve package from spec or package, falling back on name and path
return f(*args, **kwds)

@Jiadalee
Copy link

Hi:

I got the same question when I using python 2.7 in Windows 7. I re-unstalled and re-installed the 'deap' but still get the same error below.

image

Anyone know how to fix it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

12 participants