Problem in installing statsmodel in Fedora 17 64-bit #617

Closed
royden83 opened this Issue Jan 11, 2013 · 26 comments

Projects

None yet

5 participants

@royden83

Hi,

I tried to install statsmodel on my Fedora 17 64-bit machine and it failed. When I ran the command "easy_install -U statsmodels", it says "ImportError: statsmodels requires pandas". But pandas is already installed on my machine.

Is there any solution to this?

Regards.
Royden

@josef-pkt
Member

My guess is that easy_install is used by a different python, then the one were you have installed pandas.

You can check the log (on commandline) of easy_install to see whether it uses the same python.

Also, easy_install install the 0.5 dev version from the scipy conference. If you want the 0.4.3 release, then it should be available by a package manager, I think.

If the problem persists, then we might have to ask on the mailing list, since I don't know if any Fedora users look at the issues.

@qwhelan
qwhelan commented Jan 14, 2013

I'm seeing the same issue on CentOS. For some reason, statsmodels isn't using setuptools to check package versions and the custom code that does the check is failing. My fix was to comment out lines 50-53 and 64-71 in setup.py and add install_requires=['pandas >= 0.7.1'] to the setup() command at the end of the file.

@rgommers
Member

Can you change except on line 52 to except ImportError? I guess it's failing with a different error for some reason. Maybe an older pandas doesn't have pandas.version or something like that.

Not using setuptools is a good thing by the way. Its -U behavior is ridiculous.

@rgommers rgommers added a commit to rgommers/statsmodels that referenced this issue Jan 14, 2013
@rgommers rgommers MAINT: in setup.py, only catch ImportError for pandas.
See for example gh-617.
0197449
@josef-pkt
Member

Another guess, but I don't see a problem with pandas '0.10.0'

possible problem with version string parsing

what is

>>> from pandas.version import version as pversion
>>> pversion
'0.9.1'
>>> pandas.__version__
'0.9.1'
@qwhelan
qwhelan commented Jan 14, 2013

Making that change to line 52 fixes the issue. No error is raised.

@rgommers
Member

That's weird. Probably because you now had a newer pandas installed. Anyway, I think this issue can be closed and gh-621 merged.

@qwhelan
qwhelan commented Jan 14, 2013

Nope, pandas v0.10 has been installed the whole time and I've replicated it on multiple machines.

@rgommers
Member

Hmm, don't get the issue then, but either way it's fixed. So closing this.

@rgommers rgommers closed this Jan 15, 2013
@jseabold
Member

@qwhelan Do you recall if you were using setuptools or distribute when you saw this?

@jseabold
Member

So if you actually catch the error that's raised by from pandas.version import version ... and raise it, you get this

|9 $ sudo easy_install statsmodels-0.5.0.tar.gz 
Processing statsmodels-0.5.0.tar.gz
Running statsmodels-0.5.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-0VhNeW/statsmodels-0.5.0/egg-dist-tmp-82w3DU
error: SandboxViolation: open('/dev/null', 'w') {}

The package setup script has attempted to modify files on your system
that are not within the EasyInstall build area, and has been aborted.

This package cannot be safely installed by EasyInstall, and may not
support alternate installation locations even if you run its setup
script by hand.  Please inform the package's author and the EasyInstall
maintainers to find out if a fix or workaround is available.

This is mighty bizarre. I actually can't install any version of statsmodels now with just setuptools, including with the ImportError. And it's really unclear to me whether there's now a different local problem. I tried to remove distribute and setuptools and reinstall just setuptools. I've got to leave this for now, but I'll try to come back to it later.

@josef-pkt
Member

Can you find out which file is being modified?
It looks like calling pandas.version is triggering a write, but I don't see anything in the pandas source that should cause a version write.

If this is not easy to resolve, then we keep this only for numpy and scipy, and add pandas and patsy to the requires and let setuptools handle the dependency.
My aversion is only to packages that update numpy or scipy.

@rgommers
Member

Old issue, googling will give you 100s of hits. Just stop using easy_install, this won't be fixed any time soon:
http://bugs.python.org/setuptools/issue23
https://bitbucket.org/tarek/distribute/issue/100

By the way, older pandas does upgrade numpy. Improved in current pandas though.

@josef-pkt
Member

The difference in those threads is that it is in writing the egg files, (it looks to me)

I have no idea what open('/dev/null', 'w') {} tries to open for writing, and why it breaks while importing pandas.version

Does pip install work on the various linux versions?

I essentially don't have any problems with easy_install on Windows (*1), so we could split the recommendation for installation between
easy_install statsmodels on Windows, installs binaries, and
pip install statsmodels on Posix (if there are no pre-packaged versions)

(*1)
easy_install[.exe] packagename has a pop-up Window to ask for write permissions on Windows 7
python <path to my python>\Scripts\easy_install-script.py packagename installs without asking for permissions.

@jseabold
Member

I saw the discussion and patch for distribute. My understanding was that having a new distribute would allow easy_install to work, but I guess this isn't the case. Still unclear to me why we're running into this. I have no idea why importing pandas 0.10.1 causes this but it's really unfortunate. At one point I saw in an error message with something about trying to write to ~/.matplotlib/tmp/, which is baffling. I never remember the details after delving into setuptools/distribute nonsense, but I would like to understand this issue better for the future.

@qwhelan
qwhelan commented Jan 26, 2013

@jseabold I'll have to get back to you on Monday, as I can't check that at the moment.

@rgommers
Member

@jseabold that patch in distribute was only for the installation script of distribute itself.

@josef-pkt
Member

according to distribute 101 https://bitbucket.org/tarek/distribute/commits/48274ba52d91
and setuptools http://bugs.python.org/setuptools/issue93

dev/null should be allowed in the sandbox

@josef-pkt
Member

just a wild guess, I don't have enough understanding of the details:

pandas has a print statement in the __init__.py. Could it be that stdout is captured and redirected even though the print shouldn't be on the execution path?

@josef-pkt josef-pkt reopened this Jan 26, 2013
@josef-pkt
Member

yipee! or Oh no!

unrelated to the current issue, but I got into troubles on Windows with python 3.3

error: Setup script exited with error: SandboxViolation: open('C:\\Programs\\Python33\\libs\\libmsvcr100.def', 'w') {}
@rgommers
Member

Writing to C:\Programs is clearly wrong. Is that again easy_install only?

@josef-pkt
Member

'C:\\Programs\\Python33 is the correct path to my 32bit python 3.3 .

yes this is with easy_install,

pip install got stuck I think, stopped writing any log on command window and never finished, maybe waiting for permissions ?? (I got impatient after not seeing anything for several minutes and killed it)

python setup.py install in the source directory, or in an unzipped sdist worked without problems
(with the full python.exe path because only my default python is on the Windows
path)

@rgommers
Member

You can use pip install statsmodels --log /path/to/log/log.txt to figure out where it gets stuck.

@josef-pkt
Member

Thanks for the tip, I will try it when I set up a python 3.3 tox virtualenv, then I will need it.

@jseabold
Member

I also saw sticking the other night. I think it's trying to download from github and not being able to (?). That's why I was just packing my own tarball and installing locally.

@qwhelan
qwhelan commented Jan 30, 2013

@jseabold Looks like it was setuptools - distribute isn't installed. I used sudo pip install statsmodels to install.

@jseabold
Member

Closing this. Please reopen if this issue crops up again.

@jseabold jseabold closed this Apr 23, 2014
@PierreBdR PierreBdR pushed a commit to PierreBdR/statsmodels that referenced this issue Sep 2, 2014
@rgommers rgommers MAINT: in setup.py, only catch ImportError for pandas.
See for example gh-617.
6773acf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment