compress: reuse the zstd compressor per thread

[ThomasWaldmann]

libzstd starts a worker pool when a compression context with nb_workers > 1 is set up and stops it again when that context is freed. borg used the one-shot zstd.compress(), so that pool was built and torn down for every single chunk.

This caches the compressor per thread (keyed by level and worker count), so the pool is created once and stays alive between chunks.

Measured

Full Silesia corpus, 2MiB chunks, 4 workers, through borg's own compressor, on an M3 Pro:

level	before	after
zstd,-4	2345 MB/s	2815 MB/s	1.20x
zstd,3	1174 MB/s	1283 MB/s	1.09x

How much this is worth depends on how expensive thread creation is on the platform, so the gain will differ elsewhere.

Notes

• The stored data does not change. set_pledged_input_size() + FLUSH_FRAME produce exactly the bytes the one-shot API produced; a test asserts this against a fresh one-shot compression, and the measured compression ratios are unchanged.
• Single-threaded compression uses the same cache. On its own that is worth next to nothing (0.98 .. 1.07x measured for 16KiB .. 2MiB chunks), but it keeps compression to one code path - _decide() no longer branches. nb_workers is left unset for it, as nb_workers == 1 would mean one asynchronous worker rather than single-threaded.
• Per thread, not global, for the same reason the scratch buffer is: compressor instances are shared between threads, compressor state must not be.
• Fork handling: in the child, the cached compressors are dropped but deliberately not freed - their worker threads do not exist there, and freeing such a context makes libzstd wait for them forever. Verified both ways: without the at-fork hook, a forked child hangs as soon as it compresses; with it, it exits cleanly.
• A failed compression drops the thread's compressors, so a context left in the middle of a frame can never emit a frame continuation as the next chunk's data.
• Caching costs memory per thread that compresses: +1.6MB peak RSS for a single-threaded level 3 context, +8MB for a 4 worker one (which also keeps its job buffers).

Test plan

• 4 new tests: cache identity (per level, per worker count, per thread, 0 workers == 1), reused output == one-shot output over consecutive chunks for both the MT and the ST path, an 8-chunk concurrent roundtrip through one shared compressor instance, and below-threshold chunks caching a one-worker context.
• Full test suite on macOS: 2928 passed, 1001 skipped. The 2 failures (completion_cmd_test.py bash completion) also fail on master without this change.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.70%. Comparing base (e861c72) to head (6c2f23c).
⚠️ Report is 9 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #10326      +/-   ##
==========================================
+ Coverage   87.65%   87.70%   +0.04%     
==========================================
  Files         103      103              
  Lines       18716    18716              
  Branches     2881     2881              
==========================================
+ Hits        16405    16414       +9     
+ Misses       1609     1600       -9     
  Partials      702      702              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.