tests fail on all os.remove() on Windows

[GoogleCodeExporter]

Temporary files are locked on Windows, so they can not be removed without 
closing beforehand.

Log file of nosetests on Windows 7, VS2008, Python 2.7 attached. 

I tried to fix the error by calling .close() method on objects that opened the 
files, deleting mmaped variables and/or TIFFfile or TIFFimage instances before 
calling os.remove(). It didn't solve the issue unfortunately. The mmaped numpy 
arrays are somehow still open.

Original issue reported on code.google.com by matej.s...@gmail.com on 8 Nov 2011 at 9:06

Attachments:

• pylibtiff_nosetests.txt

[GoogleCodeExporter]

A possible workaround would be enclose .close() method calls
within try-except block. This is to see whether the tests
pass under Windows.

Next workaround would be to remove temporary files using atexit module.
Perhaps the memmap files are closed at the exit time of Python.

Original comment by pearu.peterson on 8 Nov 2011 at 9:23

• Changed state: Accepted

[GoogleCodeExporter]

You meant to enclose os.remove() calls within try-except block, don't you?

Original comment by matej.s...@gmail.com on 8 Nov 2011 at 9:28

[GoogleCodeExporter]

Yes.

Original comment by pearu.peterson on 8 Nov 2011 at 10:15

[GoogleCodeExporter]

With this patch tests proceed without error. But it's only a workaround, few 
thousands of temporary files are created...

Original comment by matej.s...@gmail.com on 8 Nov 2011 at 10:18

Attachments:

• pylibtiff_tests_windows.patch

[GoogleCodeExporter]

hundreds not thousands...

Original comment by matej.s...@gmail.com on 8 Nov 2011 at 10:20

[GoogleCodeExporter]

I just committed a patch that uses atexit to cleanup temporary files.
Let me know whether it works under Windows.

Original comment by pearu.peterson on 8 Nov 2011 at 10:53

[GoogleCodeExporter]

It doesn't work.

c:\...\libtiff\tests>nosetests -s
.....277:292,[[(250, 0, 0) (0, 251, 0) (0, 0, 252)]
 [(250, 0, 0) (0, 251, 0) (0, 0, 252)]]
[array([[[250,   0,   0],
        [250,   0,   0]]], dtype=uint8), array([[[  0, 251,   0],
        [  0, 251,   0]]], dtype=uint8), array([[[  0,   0, 252],
        [  0,   0, 252]]], dtype=uint8)]
...
----------------------------------------------------------------------
Ran 8 tests in 4.486s

OK
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "C:\Python27\lib\atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
WindowsError: [Error 32] The process cannot access the file because it is being 
used by another process: 'c:\\users\\smid\\appdata\\local\\temp\\tmpw3kmc0.tif'
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "C:\Python27\lib\atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
WindowsError: [Error 32] The process cannot access the file because it is being 
used by another process: 'c:\\users\\smid\\appdata\\local\\temp\\tmptwma83.tif'
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "C:\Python27\lib\atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
...
...
...

Maybe this can give a clue: 
http://stackoverflow.com/questions/1519956/numpy-and-memmap-errno-24-too-many-op
en-files but I do not have time to do it myself just now.

Original comment by matej.s...@gmail.com on 8 Nov 2011 at 12:06

[GoogleCodeExporter]

Ok, I'll take a look at it later. Occasionally, I have seen 
the too many open files issue when working over nfs and
the proposed clue is very interesting to me too.

Thanks for feedback so far!

Original comment by pearu.peterson on 8 Nov 2011 at 12:44

[pearu]

Fixed by b5ae731#diff-009710fe6f723d844bf272564b11e8c6
On Windows, the tiff files must be closed before removing.