Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TestProbitCG failure on Ubuntu #109

Closed
mikix opened this issue Nov 15, 2011 · 12 comments

Comments

Projects
None yet
4 participants
@mikix
Copy link

commented Nov 15, 2011

On Ubuntu Precise (haven't tested on stable version), test_discrete.py fails, causing a build failure.

https://launchpadlibrarian.net/83193398/buildlog_ubuntu-precise-i386.statsmodels_0.3.1-2_FAILEDTOBUILD.txt.gz

Simply bumping TestProbitCG's maxiter value from 250 to 500 fixes it. 500 matches the same iters that TestProbitNM is afforded, so it doesn't seem unreasonable.

@jseabold jseabold closed this in 590bd62 Nov 15, 2011

@richbwood

This comment has been minimized.

Copy link

commented Dec 19, 2012

Hi,

I am using the latest build as of 19/12/2012:
statsmodels-c9062e43b8a5f7385537ca95f875c4f19e7ac6dd.zip

TestProbitCG is failing again, even though I can confirm that maxiter is set to 500.

FAIL: statsmodels.discrete.tests.test_discrete.TestProbitCG.test_conf_int

Traceback (most recent call last):
File "/home/woodri/build/out/lib/python2.7/site-packages/nose/case.py", line 187, in runTest
self.test(*self.arg)
File "/home/woodri/build/out/lib/python2.7/site-packages/statsmodels-0.5.0-py2.7-linux-x86_64.egg/statsmodels/discrete/tests/test_discrete.py", line 46, in test_conf_int
assert_almost_equal(self.res1.conf_int(), self.res2.conf_int, DECIMAL_4)
File "/home/woodri/build/out/lib/python2.7/site-packages/numpy/testing/utils.py", line 452, in assert_almost_equal
return assert_array_almost_equal(actual, desired, decimal, err_msg)
File "/home/woodri/build/out/lib/python2.7/site-packages/numpy/testing/utils.py", line 800, in assert_array_almost_equal
header=('Arrays are not almost equal to %d decimals' % decimal))
File "/home/woodri/build/out/lib/python2.7/site-packages/numpy/testing/utils.py", line 636, in assert_array_compare
raise AssertionError(msg)
AssertionError:
Arrays are not almost equal to 4 decimals

(mismatch 12.5%)
x: array([[ 0.26584602, 2.98583433],
[ -0.11269245, 0.21615338],
[ 0.26008598, 2.5926075 ],
[-12.43566596, -2.46925458]])
y: array([[ 0.2658255, 2.985795 ],
[ -0.1126929, 0.2161508],
[ 0.2600795, 2.592585 ],
[-12.43547 , -2.469166 ]])


Thank you.

@josef-pkt

This comment has been minimized.

Copy link
Member

commented Dec 19, 2012

which scipy and numpy versions are you using?

Since this tests is skipped on Windows, maybe there are more serious problems with fmin_cg and we should rethink what to do with this test.
One change that might have affected this is that we are now renormalizing the likelihood function by 1/nobs for the optimization.

@josef-pkt

This comment has been minimized.

Copy link
Member

commented Dec 19, 2012

can you try to lower gtol to gtol=1e-8 and run the tests?

@josef-pkt

This comment has been minimized.

Copy link
Member

commented Dec 19, 2012

It looks like there is still a version issue.
This test doesn't fail on Travis-CI nor in the pythonxy Ubuntu tests https://code.launchpad.net/~pythonxy/+recipe/statsmodels-daily-current

@josef-pkt josef-pkt reopened this Dec 19, 2012

@richbwood

This comment has been minimized.

Copy link

commented Dec 19, 2012

uname -a
Linux xldn5103pap 2.6.18-194.11.4.el5 #1 SMP Fri Sep 17 04:57:05 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux

numpy.version
'1.6.2'

scipy.version
'0.11.0'

Changing gtol=1e-8 fixed the tests:

Ran 1857 tests in 244.987s

OK (SKIP=12)

@josef-pkt

This comment has been minimized.

Copy link
Member

commented Dec 19, 2012

Good, lowering gtol compensates for the normalization, and requires roughly the same precision as before.

Thank you.

@josef-pkt

This comment has been minimized.

@josef-pkt

This comment has been minimized.

Copy link
Member

commented Jan 14, 2013

correction: that builds statsmodels_0.4.2-1
There shouldn't be a failure with that. ?

@jseabold

This comment has been minimized.

Copy link
Member

commented Jan 14, 2013

Would this fail in master? Should we focus on getting a release out? Features / clean up / PR review is a neverending job...

@josef-pkt

This comment has been minimized.

Copy link
Member

commented Jan 14, 2013

Yes, it's still on current master, just lowering the gtol should be enough.

I wasn't set up for changing master at the time and had forgotten about it.

I can start in 2 weeks with release preparation, then I have hopefully larger time blocks available.
I want to go through the errors in
#613
I saw several bugs in the code (in untested code paths) where I would also like to check why they are not tested.

@josef-pkt josef-pkt referenced this issue Feb 3, 2013

Merged

Misc fixes 05 #640

@josef-pkt josef-pkt closed this in 44a93ea Feb 20, 2013

@josef-pkt

This comment has been minimized.

Copy link
Member

commented Apr 26, 2014

test failure on Windows 64bit MingW binaries
http://mail.scipy.org/pipermail/numpy-discussion/2014-April/070043.html
maxiter is too low, needs 766 iterations instead of less than maxiter=500, but then the result has atol 1e-6 instead of 4 decimals as in test suite.

looks like fmin_cg needs larger maxiter

@josef-pkt

This comment has been minimized.

Copy link
Member

commented Jun 18, 2014

see new issue #1690
fixed again in PR #1699 and test failures on some machines in PR #1766

Unit tests are just working around the problems in scipy's fmin_cg.

Problems in using fmin_cg in "not nice" cases still persist. Just make the estimation problem "nice".

@josef-pkt josef-pkt closed this Jun 18, 2014

PierreBdR pushed a commit to PierreBdR/statsmodels that referenced this issue Sep 2, 2014

PierreBdR pushed a commit to PierreBdR/statsmodels that referenced this issue Sep 2, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.