Skip to content

4.3.0

Latest

Choose a tag to compare

@github-actions github-actions released this 22 Nov 02:15
Immutable release. Only release title and notes can be modified.

Features

  • On Windows, release_with_timeout limits the caller's wait for release. If it raises ReleasePendingError, call wait on the exception from any thread for the result. On POSIX, release is synchronous (#1107) by @gaborbernat. (#1100)

Bug fixes

  • lock_descriptor raises TypeError unless blocking is a bool (#969) by @feiiiiii5 and @gaborbernat.
  • On POSIX filesystems with atomic exchange, SoftFileLock restores a live marker after an interrupted exchange and removes its own unfinished marker during cleanup (#1080) by @gaborbernat. (#993)
  • Unix soft fallback markers exclude group and other writes even on filesystems without atomic exchange (#1057) by @gaborbernat. (#994)
  • SoftFileLock, MarkerSoftFileLock and SoftFileLease remove empty markers after a signal interrupts marker creation or stale-marker replacement (#1049) by @gaborbernat. (#996)
  • Unix locks select the soft fallback per resolved path, including after a directory change. The lock keeps its Unix type and supports pid, is_lock_held_by_us, break_lock() and lifetime during fallback (#1057) by @gaborbernat. (#1000)
  • On POSIX, ReadWriteLock and AsyncReadWriteLock raise RuntimeError in detected subinterpreters, including InterpreterPoolExecutor workers and mod_wsgi or uWSGI subinterpreters. Use the main interpreter or a separate process. Detection on embedded CPython 3.11 requires _xxsubinterpreters or ctypes access to the interpreter C API (#1106) by @gaborbernat. (#1003)
  • On Windows, SoftFileLock treats a symlink as an occupied marker and leaves its target untouched (#1049) by @gaborbernat. (#1009)
  • SoftFileLock and SoftFileLease clean up markers and descriptors across interrupted acquires and releases. POSIX unlink errors propagate, with a retry after close on filesystems that require it. Retry release if a timeout interrupts restoration of a displaced marker (#1062) by @gaborbernat. (#1014)
  • A singleton shared by relative and absolute paths refuses acquisition if a directory change redirects its stored lock path (#1098) by @gaborbernat. (#1018)
  • ReadWriteLock raises RuntimeError if a directory change redirects a singleton's cached database path (#1058) by @gaborbernat. (#1019)
  • SoftFileLock, MarkerSoftFileLock and SoftFileLease remove their pending recovery marker even if another process replaces the break name during rollback (#1082) by @gaborbernat. (#1025)
  • Single-threaded SQLite read and write lock waits allow Python signal handlers to run. Waiting writers keep their place ahead of new readers (#1105) by @gaborbernat. (#1027)
  • On POSIX, SoftFileLock tries to leave a one-byte recovery record if a full filesystem prevents marker cleanup. With atomic exchange, a later caller can reclaim it when space is available (#1101) by @Yshuro and @gaborbernat. (#1029)
  • Windows soft-lock recovery no longer reads proc/self/ns/pid from the drive root when identifying a marker's host (#1036) by @gaborbernat.
  • On Unix filesystems without flock, waiters can reclaim an interrupted acquire's recovery marker after two seconds if atomic exchange and space for the marker are available (#1057) by @gaborbernat. (#1046)
  • SoftReadWriteLock recreates a missing generation directory during read or write acquisition, even if a holder record survives (#1103) by @gaborbernat. (#1102)
  • On POSIX, soft locks close the inspection descriptor if opening an existing marker is interrupted (#1113) by @gaborbernat. (#1111)
  • Unix locks can use the soft fallback when the directory denies the temporary probe and atomic exchange is unavailable. If native locking works there, new lock files exclude group and other writes by default. Set mode=0o660 for group sharing. The probe removes its temporary name before testing and closes its descriptor on interruption (#1115) by @gaborbernat. (#1112)
  • ReadWriteLock prevents database redirection in shared, non-sticky directories by placing its hard link in a fresh private directory. Cleanup recognizes older link layouts (#1096) by @gaborbernat.
  • AsyncFileLock, AsyncReadWriteLock and AsyncSoftReadWriteLock discard executor calls whose submission failed. A queued release can no longer drop a later task's lock (#986) by @gaborbernat.
  • SoftReadWriteLock scans up to 256 orphan holder records and 256 temporary files per sweep, resuming where it stopped. Each listing removes up to 256 obsolete generations. The lock raises a protocol error above 65,536 generation entries (#1097) by @gaborbernat.
  • SoftReadWriteLock uses fixed-size fingerprints to bound memory use per holder record during cleanup (#1097) by @gaborbernat.
  • StrictSoftFileLock backs off above 1,024 public claims and raises SoftFileLockProtocolError above 16,380 directory entries before publication. Sentinel cleanup counts only the lock's own records (#1040) by @gaborbernat.
  • StrictSoftFileLock creates sentinels and claims without group or other write permissions. Run chmod go-w on older writable sentinels. On Linux, a contender waits if fs.protected_hardlinks prevents it from probing another account's claim (#985) by @gaborbernat.
  • Timeout and lock_path_occupant now escape control characters and other bytes outside printable non-space ASCII in recorded hostnames (#1090) by @radhika1314 and @gaborbernat.
  • A SoftReadWriteLock heartbeat checks the holder token before writing. If another file replaces the record and reuses its inode, the lock reports a lost hold (#982) by @gaborbernat.
  • A SoftReadWriteLock waiter paused past stale_threshold gets a fresh token so cleanup of its old record cannot remove the new hold (#1097) by @gaborbernat.
  • A failed acquire rolls back its own nesting level. Releasing that level from a hook or signal handler raises RuntimeError. Balanced nested holds still work. Retrying an interrupted release on a shared instance affects only the original hold. An indefinite acquire raises RuntimeError if a signal handler returns holding the same path through another instance (#1060) by @gaborbernat.
  • A forked child with its parent's numeric PID in another Linux PID namespace preserves the parent's locks and private SQLite database link (#1075) by @gaborbernat.
  • After an ambiguous Unix unlock error, FileLock rejects reentrant acquires until an explicit release() retry resolves the hold (#1098) by @gaborbernat.
  • Async reader/writer wrappers sharing a lock serve waiting writers in arrival order, ahead of new readers. Returning writers rejoin at the back (#1073) by @gaborbernat.
  • At interpreter exit, SoftReadWriteLock keeps holds owned by live daemon threads. AsyncSoftReadWriteLock keeps holds whose event loops are still running (#1048) by @gaborbernat.
  • Canceling an AsyncFileLock release still decrements the lock counter after the active transition finishes (#1051) by @gaborbernat.
  • Closing an async reader/writer wrapper stops its contended acquires on the next poll, even if another singleton wrapper is waiting (#1073) by @gaborbernat.
  • Closing an async reader/writer wrapper wakes tasks waiting on that wrapper even if another singleton wrapper holds the lock (#1073) by @gaborbernat.
  • During shutdown, another thread's release(force=True) preserves the main thread's thread-local hold, including during atexit callbacks (#1098) by @gaborbernat.
  • If an async reader/writer acquire succeeds during close, it waits for close to finish. A successful close releases the hold and the acquire raises the closed error. If close fails while the lock is held, the task receives the hold (#1073) by @gaborbernat.
  • In a sticky Unix directory such as /tmp, ReadWriteLock raises PermissionError unless the caller or directory owner owns the database (#970) by @radhika1314 and @gaborbernat.
  • In sticky directories, locks reject files and protocol directories whose ownership is ambiguous because Linux reports the overflow UID (#977) by @gaborbernat.
  • Linux accounts without CAP_FOWNER can use their own native, strict soft and soft reader/writer locks in sticky directories even with an overflow UID or unreadable procfs. This includes OverlayFS containers. Use a private directory if ownership cannot be verified (#1099) by @gaborbernat.
  • Linux soft locks ignore a mounted-over boot ID and compare process start times, preserving live markers across mounts (#1063) by @gaborbernat.
  • On Linux, ReadWriteLock and AsyncReadWriteLock honor the acquire timeout without busy-waiting on a leased database. Nonblocking acquires return promptly (#1059) by @gaborbernat.
  • On Linux, SoftFileLock, MarkerSoftFileLock and SoftFileLease use only the PID to check liveness when the holder or contender sees another PID namespace's /proc. This preserves live markers (#983) by @gaborbernat.
  • On Linux, a soft lock held by a worker thread no longer admits another holder when the main thread exits inside a time namespace (#1054) by @gaborbernat.
  • On POSIX and Windows, ReadWriteLock opens its database through a private hard link if a -journal, -wal or -shm file already exists, preventing replay of a leftover journal (#976) by @gaborbernat.
  • On POSIX systems with atomic name exchange, SoftFileLock preserves the live holder's marker when recovery encounters another account's record in a sticky directory (#975) by @gaborbernat.
  • On POSIX systems with atomic name exchange, SoftFileLock waits for a live stale-lock breaker before recovering its marker (#974) by @gaborbernat.
  • On POSIX, StrictSoftFileLock rejects another account's compatibility sentinel in a sticky directory, preserving exclusion with SoftFileLock (#992) by @gaborbernat.
  • On POSIX, ReadWriteLock and AsyncReadWriteLock preserve mutual exclusion if their private link disappears before SQLite opens it. Linux cleanup preserves peer links when procfs cannot identify their PID namespace (#1096) by @gaborbernat.
  • On POSIX, ReadWriteLock and AsyncReadWriteLock remove dead holders' empty link directories after verifying the link belongs to the current database (#973) by @gaborbernat.
  • On POSIX, SoftFileLease now reports a missing marker when another hard link keeps its old inode alive (#1069) by @gaborbernat.
  • On POSIX, SoftFileLease records claim loss detected during release in compromise. Only the heartbeat thread calls on_compromise (#1069) by @gaborbernat.
  • On Windows, ReadWriteLock rejects junctions in its private database alias path and prevents directory renames until SQLite opens the database (#1079) by @gaborbernat.
  • On Windows, SoftFileLock rechecks its acquire deadline after each half-second attempt to rename a stale marker held by a reader's oplock (#984) by @gaborbernat.
  • On Windows, StrictSoftFileLock honors its acquire timeout when an oplock blocks the sentinel. On Linux, a sentinel lease counts as contention after a short grace period, including for blocking=False (#987) by @gaborbernat.
  • On Windows, StrictSoftFileLock honors the acquire timeout while pinning directories, publishing claims or withdrawing after a lost race. A worker finishes delayed withdrawal. Interrupted publication can leave private records requiring manual removal. Release waits for Windows to close both public claims (#989) by @gaborbernat.
  • On macOS and systems with CLOCK_BOOTTIME, SoftReadWriteLock includes system sleep when checking waiter leases and renews expired tokens before entry (#1097) by @gaborbernat.
  • Outside Windows, StrictSoftFileLock rejects a directory with unrelated entries at a protocol path and preserves its permissions and contents (#978) by @gaborbernat.
  • Under a sticky parent, SoftReadWriteLock rejects a <path>.rw tree, gen or holders writable by other accounts and rechecks subdirectory ownership on open. Remove unsafe trees so the lock can recreate them privately (#988) by @gaborbernat.
  • AsyncReadWriteLock can close after the default executor shuts down. Closing either async reader/writer lock again after a canceled close waits for its worker to exit (#1110) by @gaborbernat.
  • AsyncSoftReadWriteLock uses a private worker by default, so a blocked acquire cannot delay unrelated releases. Awaiting close() joins the worker (#1073) by @gaborbernat.
  • ReadWriteLock and AsyncReadWriteLock reject an acquire if SQLite switches to WAL mode after the header check (#980) by @gaborbernat.
  • ReadWriteLock and AsyncReadWriteLock reject unsupported SQLite file-format versions that could undermine write locking (#979) by @gaborbernat.
  • ReadWriteLock and AsyncReadWriteLock reject write locks on read-only SQLite connections (#1095) by @gaborbernat.
  • SoftFileLease and AsyncSoftFileLease reject reentrant acquisition after the heartbeat detects a lost claim. Release the lost hold before acquiring again (#1098) by @gaborbernat.
  • SoftReadWriteLock and AsyncSoftReadWriteLock limit generation scans during acquire. Release raises a protocol error if it cannot withdraw the holder before its stale threshold (#981) by @gaborbernat.
  • SoftReadWriteLock and AsyncSoftReadWriteLock preserve active readers when a delayed writer tries to reuse the first generation. Acquires retry if the log resets before publication (#1074) by @gaborbernat.

Documentation

  • A forked child closes the inherited FileLock descriptor. Daemons must create and acquire a new lock after forking (#1053) by @gaborbernat. (#1028)
  • Explain how to reacquire a Windows SoftFileLock when marker deletion is still pending after release (#1108) by @gaborbernat. (#1086)
  • Document how to recover a soft lock when a process reuses the PID of a dead breaker (#1094) by @gaborbernat.
  • The API reference now documents lock policy types, acquisition proxies, reader/writer settings and asyncio lock contexts (#1122) by @gaborbernat.

Security

  • ReadWriteLock and AsyncReadWriteLock limit SQLite schema loading. Acquiring a database with a large schema can raise ValueError (#1035) by @gaborbernat.
  • ReadWriteLock and AsyncReadWriteLock limit the directory entries scanned for dead holders when opening a connection (#1096) by @gaborbernat.
  • A failed AsyncReadWriteLock construction no longer prevents a forked child from resetting other locks (#1031) by @gaborbernat.
  • Canceling an AsyncReadWriteLock acquire on Windows now ends its wait for a database oplock (#1059) by @gaborbernat.
  • On Unix, acquiring a lock with an on_acquired callback raises PermissionError when the lock file has another hard link and another account can write its directory. Without a callback, or in a directory only the caller can write, hard-linked lock names still share the same native lock (#1118) by @radhika1314 and @gaborbernat.
  • On Windows, StrictSoftFileLock skips abandoned private records to avoid waiting on reader oplocks. For manual cleanup, see how-to:Use fail-closed soft locks (#1045) by @gaborbernat.
  • On Windows, FileLock treats a pending oplock break as contention until the acquire deadline. acquire(blocking=False) raises Timeout immediately (#1033) by @gaborbernat.
  • Strict locks publish hard links without following symlinks even when the filesystem capability probe cannot run (#990) by @gaborbernat.

Performance

  • SoftReadWriteLock reuses the generation directory handle to reduce ownership checks during write acquisition and release (#1114) by @gaborbernat.
  • Reduce the cost of reentrant acquire and release (#1116) by @gaborbernat.
  • Speed up stale lock recovery for SoftFileLock in directories with many unrelated files (#1092) by @gaborbernat.
  • SoftReadWriteLock backs off for up to 10 ms when a peer publishes first during withdrawal, reducing filesystem traffic (#1091) by @gaborbernat.

New Contributors

🐍 PyPI release

📜 Full Changelog