test_arima.test_small_data: SVD fails to converge (Python 3.4 / Ubuntu 12.04) #1689

andrewclegg opened this Issue May 24, 2014 · 4 comments


None yet

2 participants


I am attempting to install statsmodels master on Python 3.4 for the Snake Charmer project. Several tests are failing or throwing errors.

Python 3.4
Ubuntu 12.04 64 bit (virtual)
NumPy 1.8.1
SciPy 0.13.1

statsmodels revision 3adaa1a
Snake Charmer revision snake-charmer-devs/snake-charmer@72734de

To reproduce: install Snake Charmer, spin up a VM as directed in the README, log in using vagrant ssh, test as normal.

ERROR: statsmodels.tsa.tests.test_arima.test_small_data
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/nose/case.py", line 198, in runTest
  File "/usr/local/lib/python3.4/dist-packages/statsmodels/tsa/tests/test_arima.py", line 1886, in test_small_data
    res = mod.fit(disp=0, start_params=[.1, .1, .1, .1])
  File "/usr/local/lib/python3.4/dist-packages/statsmodels/tsa/arima_model.py", line 886, in fit
    callback=callback, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/statsmodels/base/model.py", line 390, in fit
  File "/usr/local/lib/python3.4/dist-packages/statsmodels/base/optimizer.py", line 184, in _fit
  File "/usr/local/lib/python3.4/dist-packages/statsmodels/base/optimizer.py", line 378, in _fit_lbfgs
  File "/usr/local/lib/python3.4/dist-packages/scipy/optimize/lbfgsb.py", line 185, in fmin_l_bfgs_b
  File "/usr/local/lib/python3.4/dist-packages/scipy/optimize/lbfgsb.py", line 314, in _minimize_lbfgsb
    f, g = func_and_grad(x)
  File "/usr/local/lib/python3.4/dist-packages/scipy/optimize/lbfgsb.py", line 258, in func_and_grad
    f = fun(x, *args)
  File "/usr/local/lib/python3.4/dist-packages/statsmodels/base/model.py", line 369, in <lambda>
    f = lambda params, *args: -self.loglike(params, *args) / nobs
  File "/usr/local/lib/python3.4/dist-packages/statsmodels/tsa/arima_model.py", line 696, in loglike
    return self.loglike_kalman(params, set_sigma2)
  File "/usr/local/lib/python3.4/dist-packages/statsmodels/tsa/arima_model.py", line 706, in loglike_kalman
    return KalmanFilter.loglike(params, self, set_sigma2)
  File "/usr/local/lib/python3.4/dist-packages/statsmodels/tsa/kalmanf/kalmanfilter.py", line 649, in loglike
    R_mat, T_mat)
  File "kalman_loglike.pyx", line 342, in statsmodels.tsa.kalmanf.kalman_loglike.kalman_loglike_double (statsmodels/tsa/kalmanf/kalman_loglike.c:5046)
  File "kalman_loglike.pyx", line 74, in statsmodels.tsa.kalmanf.kalman_loglike.kalman_filter_double (statsmodels/tsa/kalmanf/kalman_loglike.c:2525)
  File "/usr/local/lib/python3.4/dist-packages/numpy/linalg/linalg.py", line 1585, in pinv
    u, s, vt = svd(a, 0)
  File "/usr/local/lib/python3.4/dist-packages/numpy/linalg/linalg.py", line 1327, in svd
    u, s, vt = gufunc(a, signature=signature, extobj=extobj)
  File "/usr/local/lib/python3.4/dist-packages/numpy/linalg/linalg.py", line 99, in _raise_linalgerror_svd_nonconvergence
    raise LinAlgError("SVD did not converge")
numpy.linalg.linalg.LinAlgError: SVD did not converge
@andrewclegg andrewclegg referenced this issue in snake-charmer-devs/snake-charmer May 24, 2014

One problem in statsmodels #14


(I thought I added a comment but I don't find it anymore.)

This is another case where we are just add the edge of what should work numerically (even if it doesn't make much sense).
We had some problems with this some time ago, but it also has been quite for a while, no test failures or errors on Ubuntu, Debian or Windows.

Since I cannot replicate this, can you try to change one of the start_params and see if it helps
on line
res = mod.fit(disp=0, start_params=[.1, .1, .1, .1])
to this
res = mod.fit(disp=0, start_params=[np.mean(y), .1, .1, .1])

adding np.mean(y) as starting value for the constant.

one common source for SVD did not converge are nans in the array, but I don't know if it would help to find them since this is not a regular use case.


np.mean(y) fixed it.


Good, that's an easy change, and doesn't make the test meaningless.

You could make a PR, or I'll fix it later today.

@andrewclegg andrewclegg added a commit to andrewclegg/statsmodels that referenced this issue May 27, 2014
@andrewclegg andrewclegg Fix for statsmodels/statsmodels#1689 cacc4e0

There you go! Thanks for the tip, glad it was a quick change.

@josef-pkt josef-pkt closed this in 0b689a2 May 27, 2014
@PierreBdR PierreBdR pushed a commit to PierreBdR/statsmodels that referenced this issue Sep 2, 2014
@andrewclegg andrewclegg Fix for statsmodels/statsmodels#1689 07da936
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment