Store: serialize all operations with an internal lock, fixes #206

[ThomasWaldmann]

Fixes #206.

A Store instance can now be shared between threads: all operations are serialized by an internal RLock (reentrant, because operations nest: create_levels uses with self:, load/store/… call find). This protects both the backends (one paramiko sftp session / one requests session are not thread-safe) and the Store's own bookkeeping (_stats Counter read-modify-writes, writethrough cache accounting).

Design points, as discussed in #206:

• list() stays a lazy generator: the lock is only held while fetching the next item, not across the whole iteration — so other threads' operations interleave with a long listing, and the iterating thread itself can do store operations inside its listing loop without deadlocking on the lock.
• Unconditional, not opt-in: an uncontended threading.RLock costs ~100 ns per acquire, noise next to any backend call (even posixfs).
• Per-operation serialization only: multi-operation sequences that need atomicity against other threads must still be coordinated by the caller (unchanged, now documented in the class docstring).
• The lock is created first in __init__ because some @_locked methods (set_levels) already run during construction.

Tests (tests/test_threading.py):

• SerializationAssertingBackend: wraps posixfs, flags any overlapping backend calls (with a small sleep to widen the race window). Verified that the hammer test does trigger violations when the lock is neutered, so the test is load-bearing.
• 4-thread store/load/info/list/delete hammer → zero overlaps.
• list() laziness: same-thread ops inside the listing loop + another thread's ops interleaving with a long listing.
• stats lost-update test: exact call counts after concurrent hammering.

Motivation: borgbackup/borg#9988 — borg2's PackWriter gets a background store-thread that stores full packs while the main thread assembles the next pack and keeps using the same Store (lock refresh, reads of already stored packs, index writes). With this released, borg can drop its interim SerializedStore wrapper and just bump its borgstore pin.

(implemented by Claude, reviewed by TW)

🤖 Generated with Claude Code