New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize failure on Windows #467

Closed
matthew-brett opened this Issue Nov 15, 2014 · 7 comments

Comments

Projects
None yet
2 participants
@matthew-brett
Member

matthew-brett commented Nov 15, 2014

Windows buildbot is green but failing: http://nipy.bic.berkeley.edu/builders/dipy-py2.7-win32/builds/97/steps/shell_8/logs/stdio

======================================================================
FAIL: dipy.core.tests.test_optimize.test_optimize
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\buildslaves\win7-32-mini\dipy-py2_7-win32\build\venv\lib\site-packages\nose\case.py", line 197, in runTest
    self.test(*self.arg)
  File "c:\buildslaves\win7-32-mini\dipy-py2_7-win32\build\venv\lib\site-packages\dipy\core\tests\test_optimize.py", line 78, in test_optimize
    assert_equal(os.path.isfile(fname), False)
  File "C:\Python27\lib\site-packages\numpy\testing\utils.py", line 317, in assert_equal
    raise AssertionError(msg)
AssertionError: 
Items are not equal:
 ACTUAL: True
 DESIRED: False
-------------------- >> begin captured stdout << ---------------------
  status: 0
 success: True
    nfev: 15
     fun: 1.1908915926500003e-16
       x: array([ -6.30050684e-09,  -6.30050684e-09,  -6.30050684e-09])
 message: 'CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL'
     jac: array([ -2.60101368e-09,  -2.60101368e-09,  -2.60101368e-09])
     nit: 2
7
5.1065423139e-11
48
[[  3.39181411e-01   6.69590705e-01   8.67836300e-01   4.33918138e+00]
 [ -3.61771467e-01  -1.99407600e-01   3.95791720e-01   1.97895841e+00]
 [ -1.12100687e-01  -2.09475587e-01   6.90915716e-02   3.45457675e-01]
 [ -4.15070644e-02   8.90949532e-02  -6.36409362e-03  -3.18205478e-02]
 [  1.13938889e-02  -7.77490684e-03  -9.08172032e-04  -4.54090873e-03]
 [  1.30668152e-04   1.25553003e-04  -2.92976602e-05  -1.46520474e-04]
 [  3.15658184e-06   9.06518691e-06  -4.53327523e-08  -2.46945128e-07]]
2
4.17603241701e-30
71
Optimization terminated successfully.
@Garyfallidis

This comment has been minimized.

Member

Garyfallidis commented Nov 18, 2014

Hi @matthew-brett,

It seems the test fails in line 78 of test_optimize

    tmp_files = opt.tmp_files
    del opt

    for fname in tmp_files:
        assert_equal(os.path.isfile(fname), False)

When opt (Optimizer object) is created it creates some temporary files for saving the evolution of
the optimizer.
So, what this is telling us is that even though I have deleted the object some of the files are still there.
But if you see at the end of the optimize.py file I do delete the temporary files using:

def __del__(self):

    for fname in self.tmp_files:
        os.remove(fname)

I am not sure how to proceed. You have played with this problems before any suggestions for a safer deletion in Win32?

Also why the builder is green although it has errors?

@matthew-brett

This comment has been minimized.

Member

matthew-brett commented Nov 18, 2014

I don't know why the test is green, I will investigate (not looking forward to that).

Is there any reason to use files for tracking the progress? Can't you use lists of strings instead?

Do you want me to submit a PR for that?

@Garyfallidis

This comment has been minimized.

Member

Garyfallidis commented Nov 19, 2014

No you don't need to submit a PR. You can just show me how you do it. That is better so that I can learn. I am impressed that the files still stay there in old windows. I thought I found a good advice on how to remove files upon object deletion. I am also considering removing saving the evolution files. But anyway if you know the trick please share your knowledge. Thx @matthew-brett !

matthew-brett added a commit that referenced this issue Nov 19, 2014

Merge pull request #469 from Garyfallidis/fix_evolution_save_win32
MRG: Fix evolution save win32

Use lists instead of files to track progress of optimizer.

This should resolve issue #467.
@matthew-brett

This comment has been minimized.

Member

matthew-brett commented Nov 23, 2014

As I predicted, the buildbot being green was a nightmare to debug, result here: scipy/scipy#4185

@matthew-brett

This comment has been minimized.

Member

matthew-brett commented Nov 23, 2014

Incidentally, the windows 32 bit script test failures are caused by the same problem.

@matthew-brett

This comment has been minimized.

Member

matthew-brett commented Nov 23, 2014

Installed scipy from a different scipy binary - red again:

http://nipy.bic.berkeley.edu/builders/dipy-py2.7-win32/builds/107

@matthew-brett

This comment has been minimized.

Member

matthew-brett commented Dec 30, 2014

I think this is closed by #469

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment