Unexpected and different behaviors on Windows and Linux

[kmdouglass]

Hi there,
When I run the following script, it runs successfully on Linux but fails on Windows 10:

import filelock

lock = filelock.FileLock('test.txt')
with lock:
    f = open('test.txt', 'w')
    f.write('hello')
    f.close()

The error on Windows is:

PermissionError: [Errno 13] Permission denied

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "lock.py", line 9, in <module>
    f.close()
PermissionError: [Errno 13] Permission denied

However, if I move f.close() outside of the context manager, the script successfully runs on Windows and Linux:

lock = filelock.FileLock('test.txt')
with lock:
    f = open('test.txt', 'w')
    f.write('hello')
f.close()

Why exactly must the f.close() be placed outside the context manager on Windows, but not on Linux? Is it related to msvcrt.locking()?

I'm using filelock 2.0.7, Windows 10 (x64) and Debian Jessie (x64).

[suresttexas00]

have you tried changing it to a "with - open" (and let python do the closing instead)? Like:

lock = filelock.FileLock('test.txt')
with lock:
    with open('test.txt', 'w') as f
        f.write('hello')

Not sure if that works here, just an idea..?

[suresttexas00]

Also curious... You are creating a lock to overwrite an existing file (versus 'a')?

[kmdouglass]

That's an interesting suggestion, @suresttexas00

Nesting the context managers also produces the permission error on a Windows machine:

import filelock

lock = filelock.FileLock('test.txt')
with lock:
    with open('test.txt', 'w') as f:
        f.write('hello')

PermissionError: [Errno 13] Permission denied

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "lock.py", line 6, in <module>
    f.write('hello')
PermissionError: [Errno 13] Permission denied

However, it works exactly as expected in Debian Jessie, producing the .txt file and raising no error. I can also confirm that this script raises the PermissionError in Windows Server 2012 R2, so it's not necessarily a problem with only Windows 10.

In my original post I was careful to make sure that test.txt was absent before every time I ran the script. Changing the write mode to a does not produce any change in the results.

This thread is actually the result of me reducing a much larger problem down to the two examples I gave in the original post. My guess is that I am making a bad assumption about how filelock should work, but I do find the inconsistency between Windows and Linux a bit surprising.

[kmdouglass]

I forgot to mention anywhere that I'm using Python 3.5.

[AdorablePotato]

Hi,
please take a look at #9 (comment).

You are probably looking for something like https://pypi.python.org/pypi/fasteners or https://pypi.python.org/pypi/lockfile/0.12.2.

[kmdouglass]

Hi @benediktschmitt ,
Thanks for your response. I think I am beginning to see where the flaw in my reasoning is.

The argument to FileLock in the line lock = filelock.FileLock('test.txt') should not be the name of the file that I want to prevent concurrent access to. Rather, it should be the name of the lock file itself, correct?

In other words, I should be doing something like:

lock = filelock.FileLock('test.txt.lock')
with lock:
    with open('test.txt', 'a') as f
        f.write('hello')

This means that it is my responsibility to write the logic that detects whether a lock file is present. Am I understanding things correctly?

Thanks a lot for your feedback.

[kmdouglass]

Hi @benediktschmitt ,
I worked through the test.py file and now have a good understanding of how py-filelock is supposed to work. Therefore, I will close this issue as a "user error."

Thanks again!

[andyneff]

I ran into this same problem in a slightly different way. Different was I wasn't opening the lock file twice:

import os

import filelock

lock = filelock.FileLock('test.txt')
with lock:
    os.write(lock._lock_file_fd, str(os.getpid()).encode())

And I also got a Permission Denied error (on unlock)

  File "C:\Users\andy\src\terra_dsm\.windows\venv\terra_dsm-h3BfiDZJ\lib\site-packages\filelock\_api.py", line 200, in release
    self._release()
  File "C:\Users\andy\src\terra_dsm\.windows\venv\terra_dsm-h3BfiDZJ\lib\site-packages\filelock\_windows.py", line 39, in _release
    msvcrt.locking(fd, msvcrt.LK_UNLCK, 1)
PermissionError: [Errno 13] Permission denied

It turns out, locking in Windows is byte based, and not file based like I would have thought.

py-filelock -> python -> ms docs

States that it is locking "bytes in a file" not a file, and that the file pointer is important (especially since py-filelock only locks one byte)

So in my case, the fix is to:

lock = filelock.FileLock('test.txt')
with lock:
    os.write(lock._lock_file_fd, str(os.getpid()).encode())
    os.lseek(lock._lock_file_fd, 0, 0)
    # Do stuff

So now if something really bad like a segfault happens, the lock file is left behind with the PID number in it (even though I lseeked, although I only verified that on Windows) and my cleanup code can verify that the pid is gone (without the need for a second file).