New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Buildbot failures with missing 'nit' key in dipy.core.optimize #461

Closed
matthew-brett opened this Issue Nov 13, 2014 · 4 comments

Comments

Projects
None yet
2 participants
@matthew-brett
Member

matthew-brett commented Nov 13, 2014

See; http://nipy.bic.berkeley.edu/builders/dipy-py2.7-osx-10.6/builds/290/steps/shell_8/logs/stdio

Could this be an incompatibility with earlier scipy? This machine has numpy 1.7 and scipy 0.11.

======================================================================
ERROR: dipy.align.tests.test_streamlinear.test_rigid_parallel_lines
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/dipy/align/tests/test_streamlinear.py", line 74, in test_rigid_parallel_lines
    new_bundle2 = srr.optimize(bundle, bundle2).transform(bundle2)
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/dipy/align/streamlinear.py", line 344, in optimize
    mat_history, opt.nfev, opt.nit)
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/dipy/core/optimize.py", line 215, in nit
    return self.res['nit']
KeyError: 'nit'

======================================================================
ERROR: dipy.align.tests.test_streamlinear.test_rigid_partial_real_bundles
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/dipy/align/tests/test_streamlinear.py", line 123, in test_rigid_partial_real_bundles
    srm = srr.optimize(static_center, moved)
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/dipy/align/streamlinear.py", line 344, in optimize
    mat_history, opt.nfev, opt.nit)
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/dipy/core/optimize.py", line 215, in nit
    return self.res['nit']
KeyError: 'nit'
-------------------- >> begin captured stdout << ---------------------
[  88.12210846  109.13088989   81.60664368]

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: dipy.align.tests.test_streamlinear.test_stream_rigid
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/dipy/align/tests/test_streamlinear.py", line 172, in test_stream_rigid
    sr_params = srr.optimize(static, moving)
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/dipy/align/streamlinear.py", line 344, in optimize
    mat_history, opt.nfev, opt.nit)
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/dipy/core/optimize.py", line 215, in nit
    return self.res['nit']
KeyError: 'nit'

======================================================================
ERROR: dipy.align.tests.test_streamlinear.test_evolution_of_previous_iterations
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/dipy/align/tests/test_streamlinear.py", line 343, in test_evolution_of_previous_iterations
    slm = slr.optimize(static, moving)
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/dipy/align/streamlinear.py", line 337, in optimize
    if opt.evolution is not None:
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/dipy/core/optimize.py", line 237, in evolution
    history = np.loadtxt(f).reshape((self.res['nit'], self.size_of_x))
KeyError: 'nit'

======================================================================
ERROR: dipy.align.tests.test_streamlinear.test_affine_real_bundles
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/dipy/align/tests/test_streamlinear.py", line 399, in test_affine_real_bundles
    slm = slr.optimize(bundle, bundle2)
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/dipy/align/streamlinear.py", line 344, in optimize
    mat_history, opt.nfev, opt.nit)
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/dipy/core/optimize.py", line 215, in nit
    return self.res['nit']
KeyError: 'nit'
-------------------- >> begin captured stdout << ---------------------
  status: 0
    nfev: 224
 success: True
     fun: 3.4462746890369411e-07
       x: array([ -3.30742409e-05,   2.70839952e-05,  -4.50638642e-04,
         2.43647501e+00,  -1.21251485e+01,  -1.14489623e+01,
         8.13637822e-01,   9.05562696e-01,   1.02810357e+00,
        -3.94146063e-02,  -8.66503992e-02,  -1.71090548e-02])
 message: 'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'
     jac: array([  4.19777333e-05,   2.05871223e-04,  -6.98220187e-04,
        -2.40374934e-06,   7.22981419e-06,  -2.13818548e-05,
        -1.37464873e-03,   5.59955634e-04,  -2.13373593e-03,
         5.50705915e-05,  -1.42710778e-03,   1.89767024e-03])

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: dipy.core.tests.test_optimize.test_optimize
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/dipy/core/tests/test_optimize.py", line 54, in test_optimize
    print(opt.nit)
  File "/Users/buildslave/osx-10.6/dipy-py2_7-osx-10_6/build/venv/lib/python2.7/site-packages/dipy/core/optimize.py", line 215, in nit
    return self.res['nit']
KeyError: 'nit'
-------------------- >> begin captured stdout << ---------------------
  status: 0
    nfev: 3
 success: True
     fun: 1.1908915926500003e-16
       x: array([ -6.30050684e-09,  -6.30050684e-09,  -6.30050684e-09])
 message: 'CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL'
     jac: array([ -2.60101368e-09,  -2.60101368e-09,  -2.60101368e-09])

--------------------- >> end captured stdout << ----------------------

@matthew-brett matthew-brett changed the title from Buildbot failures with missing 'nit' key in scipy.optimize to Buildbot failures with missing 'nit' key in dipy.core.optimize Nov 13, 2014

@matthew-brett

This comment has been minimized.

@Garyfallidis

This comment has been minimized.

Member

Garyfallidis commented Nov 13, 2014

Hi @matthew-brett, according to
http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.optimize.minimize.html

the minimize function was introduced in 0.11.0 so it should be called here. Minimize returns a dictionary which should have a key called 'nit' (number of iterations). Could it be that the key was added after 0.11?

Ooops! Yes! Look here:
In 0.11 no 'nit' !
https://github.com/scipy/scipy/blob/maintenance/0.11.x/scipy/optimize/lbfgsb.py#L175
And in 0.12 'nit' is there
https://github.com/scipy/scipy/blob/maintenance/0.12.x/scipy/optimize/lbfgsb.py#L189

Okay will make a fix asap! Thanx for reporting. Hmm.. this is a serious bug for 0.11 actually. You cannot even set the number of maximum iterations.... Grrr....!

@Garyfallidis

This comment has been minimized.

Member

Garyfallidis commented Nov 13, 2014

Hopefully the issue is addressed here #465

matthew-brett added a commit that referenced this issue Nov 14, 2014

Merge pull request #465 from Garyfallidis/fix_optimize_scipy_version
MRG: increased SCIPY version definition flag to 0.12

Change scipy dependency rules; should fix issue #461
@matthew-brett

This comment has been minimized.

Member

matthew-brett commented Nov 15, 2014

Fixed by #465 - thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment