In IPython for Python 3, system site-packages comes before user site-packages

[astrofrog]

I noticed the following strange behavior with IPython for Python 3: if I have e.g. Numpy 1.6.2 installed in my system installation (e.g. MacPorts) then if I install the latest dev Numpy (1.8.0.dev) into my --user directory, python will correctly pick up Numpy 1.8.0.dev as having priority, but IPython does not. Here's what I get for sys.path in both:

Python:

>>> import sys
>>> sys.path
>>> import sys
>>> sys.path
['',
'/Volumes/Raptor/Library/Python/3.2/lib/python/site-packages/packagename-0.0.dev31-py3.2-macosx-10.8-x86_64.egg',
'/Volumes/Raptor/Library/Python/3.2/lib/python/site-packages/astroquery-0.0.dev218-py3.2.egg',
'/Volumes/Raptor/Library/Python/3.2/lib/python/site-packages/sedfitter-0.1.2.dev117-py3.2.egg',
'/Volumes/Raptor/Library/Python/3.2/lib/python/site-packages/astropy-0.3.dev3169-py3.2-macosx-10.8-x86_64.egg',
'/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python32.zip',
'/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2',
'/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/plat-darwin',
'/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/readline',
'/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/lib-dynload',
'/Volumes/Raptor/Library/Python/3.2/lib/python/site-packages',
'/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages',
'/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/PyObjC',
'/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/setuptools-0.6c11-py3.2.egg-info']

IPython:

['',
 '/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages',
 '/opt/local/Library/Frameworks/Python.framework/Versions/3.2/bin',
 '/Volumes/Raptor/Library/Python/3.2/lib/python/site-packages/packagename-0.0.dev31-py3.2-macosx-10.8-x86_64.egg',
 '/Volumes/Raptor/Library/Python/3.2/lib/python/site-packages/astroquery-0.0.dev218-py3.2.egg',
 '/Volumes/Raptor/Library/Python/3.2/lib/python/site-packages/sedfitter-0.1.2.dev117-py3.2.egg',
 '/Volumes/Raptor/Library/Python/3.2/lib/python/site-packages/astropy-0.3.dev3169-py3.2-macosx-10.8-x86_64.egg',
 '/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python32.zip',
 '/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2',
 '/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/plat-darwin',
 '/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/readline',
 '/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/lib-dynload',
 '/Volumes/Raptor/Library/Python/3.2/lib/python/site-packages',
 '/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/PyObjC',
 '/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/setuptools-0.6c11-py3.2.egg-info',
 '/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/IPython/extensions',
 '/Volumes/Raptor/.ipython']

This does not occur with Python 2, only Python 3. You can see that in IPython there are two extra entries at the top (starting with /opt/local/). This is causing all kinds of issues, so I think it's a bug.

Just to clarify, /opt/local/ is MacPorts (system install) and /Volumes/Raptor/ is my home directory.

[takluyver]

It's certainly a problem, but off the top of my head, it's not something we're doing. Does the hashbang of IPython definitely point to the same Python executable that you're using?

[astrofrog]

Almost - the ipython script has:

#!/opt/local/Library/Frameworks/Python.framework/Versions/3.2/Resources/Python.app/Contents/MacOS/Python

but python points to

/opt/local/Library/Frameworks/Python.framework/Versions/3.2/bin/python3.2

However, even if I start up

/opt/local/Library/Frameworks/Python.framework/Versions/3.2/Resources/Python.app/Contents/MacOS/Python

I get:

['',
'/Volumes/Raptor/Library/Python/3.2/lib/python/site-packages/packagename-0.0.dev31-py3.2-macosx-10.8-x86_64.egg',
'/Volumes/Raptor/Library/Python/3.2/lib/python/site-packages/astroquery-0.0.dev218-py3.2.egg',
'/Volumes/Raptor/Library/Python/3.2/lib/python/site-packages/sedfitter-0.1.2.dev117-py3.2.egg',
'/Volumes/Raptor/Library/Python/3.2/lib/python/site-packages/astropy-0.3.dev3169-py3.2-macosx-10.8-x86_64.egg',
'/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python32.zip',
'/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2',
'/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/plat-darwin',
'/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/readline',
'/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/lib-dynload',
'/Volumes/Raptor/Library/Python/3.2/lib/python/site-packages',
'/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages',
'/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/PyObjC',
'/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/setuptools-0.6c11-py3.2.egg-info']

so it seems that IPython is adding the two extra paths at the top. One of the weird things is that one of the paths it adds it doesn't really add, but just moves it to the top (site-packages) while the other shouldn't really even be in sys.path.

[astrofrog]

If I print sys.path at the start of /opt/local/bin/ipython, it does not add the site-packages directory (it adds the bin one because that's where the real ipython file lives I think). So it must be adding the site-packages at the start later on.

[astrofrog]

Actually, I'm having difficulties reproducing this, so ignore me for now - I will investigate further and report back

[minrk]

With no report back, shall we assume this is addressed? Go ahead and reopen if it's still an Issue.