chunkers, crypto: release the GIL in pure-C hot paths

[ThomasWaldmann]

Release the GIL in all the no-risk pure-C regions of the Cython kernels, so other threads (FUSE request handling, the lock-refresh thread, and future parallel create/transfer pipelines) are no longer blocked while borg chunks, hashes, or en/decrypts.

What

Chunkers (buzhash, buzhash64, fastcdc):

• _buzhash* / BARREL_SHIFT* helpers declared noexcept nogil (pure arithmetic, cannot raise),
• with nogil: around the byte-scan loops, the initial window hash, and fill()'s bulk memmove/memcpy/memset (the PyBytes_AsString pointer is captured while holding the GIL).

crypto/low_level.pyx:

• OpenSSL extern blocks declared nogil (declaration-only),
• GIL released around only the bulk EVP_EncryptUpdate/EVP_DecryptUpdate calls (AEAD encrypt+decrypt, AES-CTR decrypt as used by borg transfer), using an rc variable so the error check/raise stays outside the block; the small Init/ctrl/Final/AAD calls are unchanged,
• XXH64: _update and the _xxh_* inline helpers marked noexcept nogil.

Why this is safe

Every wrapped region is pure C on caller-owned buffers: no Python API use, no refcounting, no exceptions. No cipher or chunker object is used concurrently from multiple threads anywhere in borg: AEAD decrypt constructs a fresh cipher per call, the encrypt-side session cipher only runs on the single-threaded write paths, borg mount is read-only, and legacy repos cannot be mounted, so CTR decrypt only runs in single-threaded borg transfer.

Deliberately not converted: AES-CTR encrypt() (test-only), platform/*.pyx syscall wrappers (would need restructuring, not mechanical), item.pyx/hashindex.pyx (Python-object work), CSPRNG and XXH64.digest() (tiny buffers).

Measurements

Two threads doing equal work each, 256 MiB random data per thread (Apple Silicon, py311); 2T/1T wall-time ratio: 2.0 = fully serialized, 1.0 = perfectly parallel:

kernel	before	after
fastcdc chunker	2.11	1.32
buzhash64 chunker	1.98	1.28
buzhash chunker	2.04	1.23
xxh64	2.01	1.01
aes256-ocb encrypt / decrypt	2.02 / 1.98	1.04 / 1.09
chacha20-poly1305 encrypt / decrypt	2.08 / 2.04	1.07 / 1.08

Single-thread throughput is unchanged: the loop bodies are identical and the GIL is released once per scanned segment / bulk call, not per byte. The chunkers' residual ~1.3 is the per-chunk Python-level work (Chunk objects, reader), left for future work.

This is groundwork for a threaded create pipeline (#37).

Verification

• chunkers + crypto test suites pass (287 tests), create/extract/transfer/compress archiver tests pass (167 tests),
• end-to-end smoke: repo-create (chacha20-poly1305) → create → check → extract → diff OK (startup selftests exercise the changed kernels in every borg process),
• the 2-thread scaling measurements above double as proof the GIL is actually released.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.10%. Comparing base (2a5cb4c) to head (4d2e068).
⚠️ Report is 1 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #10014   +/-   ##
=======================================
  Coverage   86.10%   86.10%           
=======================================
  Files          96       96           
  Lines       17326    17326           
  Branches     2649     2649           
=======================================
  Hits        14918    14918           
  Misses       1667     1667           
  Partials      741      741           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.