Skip to content

Commit

Permalink
lmplements the inter-process readers writer lock
Browse files Browse the repository at this point in the history
  • Loading branch information
Paulius Šarka authored and Paulius Sarka committed Aug 10, 2020
1 parent 75148cb commit 6b59d8d
Show file tree
Hide file tree
Showing 10 changed files with 841 additions and 92 deletions.
2 changes: 2 additions & 0 deletions .gitignore
@@ -0,0 +1,2 @@
*.egg-info
__pycache__
54 changes: 44 additions & 10 deletions .travis.yml
@@ -1,16 +1,50 @@
language: python
python:
- "2.7"
- "3.4"
- "pypy"
# https://github.com/travis-ci/travis-ci/issues/4794
matrix:

jobs:
include:
- python: "3.5"
- name: "Python 2.7 on Linux"
python: 2.7
before_install:
- pip install .
- pip install -r test-requirements.txt

- name: "Python 3.6 on Linux"
python: 3.6
before_install:
- pip install .
- pip install -r test-requirements.txt

- name: "Python 3.8 on Linux"
python: 3.8
before_install:
- pip install .
- pip install -r test-requirements.txt

- name: "Python 3.? on macOS"
os: osx
osx_image: xcode12
language: shell
before_install:
- python3 --version
- pip3 install -U pip
- pip3 install --user .
- pip3 install --user -r test-requirements.txt
script:
- python3 -m nose

- name: "Python 3.6 on Windows"
os: windows
language: shell
before_install:
- choco install python --version 3.6
- python -m pip install --upgrade pip
- pip install .
- pip install -r test-requirements.txt
env: PATH=/c/Python36:/c/Python36/Scripts:$PATH

# command to install dependencies
install:
- "pip install ."
- "pip install -r test-requirements.txt"
- ls

# command to run tests
script: nosetests

32 changes: 21 additions & 11 deletions README.rst
Expand Up @@ -22,32 +22,42 @@ Fasteners
Overview
--------

A python `package`_ that provides useful locks.
A python `package`_ that provides cross-platform inter-thread and inter-process
locks.

It includes the following.
It includes the following:

Locking decorator
*****************
Inter-thread locking decorator
******************************

* Helpful ``locked`` decorator (that acquires instance
objects lock(s) and acquires on method entry and
releases on method exit).

Reader-writer locks
*******************
Inter-thread reader writer locks
********************************

* Multiple readers (at the same time).
* Single writers (blocking any readers).
* Single writer (blocking any readers).
* Helpful ``read_locked`` and ``write_locked`` decorators.

Inter-process locks
*******************
Inter-process locking decorator
*******************************

* Single writer using file based locking (these automatically
release on process exit, even if ``__release__`` or
* Single process lock using a file based locking that automatically
release on process exit (even if ``__release__`` or
``__exit__`` is never called).
* Helpful ``interprocess_locked`` decorator.

Inter-process reader writer lock
********************************

* Multiple readers (at the same time)
* Singer writer (blokcing any readers)
* Can be used via ``interprocess_read_locked`` and ``interprocess_write_locked``
decorators, or ``read_lock`` and ``write_lock`` context managers.
* Based on fcntl (Linux, OSX) and LockFileEx (Windows)

Generic helpers
***************

Expand Down
61 changes: 61 additions & 0 deletions doc/source/api/process_lock.rst
Expand Up @@ -2,6 +2,56 @@
Process lock
==============

Fasteners inter-process locks are cross-platform and are released automatically
if the process crashes. They are based on the platform specific locking
mechanisms:

* fcntl for posix (Linux and OSX)
* LockFileEx (via pywin32) and \_locking (via msvcrt) for Windows

The intersection of fcntl and LockFileEx features is quite small, hence you
should always assume that:

* Locks are advisory. They do not prevent the modification of the locked file
by other processes.

* Locks can be unintentionally released by simply opening and closing the file
descriptor, so lock files must be accessed only using provided abstractions.

* Locks are not reentrant_. An attempt to acquire a lock multiple times can
result in a deadlock or a crash upon a release of the lock.

* Reader writer locks are not upgradeable_. An attempt to get a reader's lock
while holding a writer's lock (or vice versa) can result in a deadlock or a
crash upon a release of the lock.

* There are no guarantees regarding usage by multiple threads in a
single process. The locks work only between processes.

To learn more about the complications of locking on different platforms we
recommend the following resources:

* `File locking in Linux (blog post)`_

* `On the Brokenness of File Locking (blog post)`_

* `Everything you never wanted to know about file locking`_

* `Windows NT Files -- Locking (pywin32 docs)`_

* `_locking (Windows Dev Center)`_

* `LockFileEx function (Windows Dev Center)`_

.. _upgradeable: https://en.wikipedia.org/wiki/Readers%E2%80%93writer_lock#Upgradable_RW_lock>
.. _reentrant: https://en.wikipedia.org/wiki/Reentrant_mutex
.. _File locking in Linux (blog post): https://gavv.github.io/articles/file-locks/
.. _On the Brokenness of File Locking (blog post): http://0pointer.de/blog/projects/locking.html
.. _Windows NT Files -- Locking (pywin32 docs): http://timgolden.me.uk/pywin32-docs/Windows_NT_Files_.2d.2d_Locking.html
.. _\_locking (Windows Dev Center): https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/locking?view=vs-2019
.. _LockFileEx function (Windows Dev Center): https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-lockfileex
.. _Everything you never wanted to know about file locking: https://chris.improbable.org/2010/12/16/everything-you-never-wanted-to-know-about-file-locking/

-------
Classes
-------
Expand All @@ -12,8 +62,19 @@ Classes
.. autoclass:: fasteners.process_lock._InterProcessLock
:members:

.. autoclass:: fasteners.process_lock.InterProcessReaderWriterLock
:members:

.. autoclass:: fasteners.process_lock._InterProcessReaderWriterLock
:members:


----------
Decorators
----------

.. autofunction:: fasteners.process_lock.interprocess_locked

.. autofunction:: fasteners.process_lock.interprocess_read_locked

.. autofunction:: fasteners.process_lock.interprocess_write_locked

0 comments on commit 6b59d8d

Please sign in to comment.