v0.3.2 - CAVLC buffer fix, int16_t compaction, and a critical thread-safety finding

Perf/correctness follow-up to v0.3.1, plus a significant finding that changes the risk picture for this driver: it has no thread synchronization anywhere, and it is exposed to that under normal use, not just under an unusual test path.

⚠️ Known critical issue: no thread synchronization, TSan-confirmed data race (unfixed)

grep -rn 'pthread_mutex\|pthread_rwlock\|atomic_' src/ returns nothing. ffmpeg calls into this driver from ≥2 of its own concurrent threads (encoder_thread/filter_thread), and ThreadSanitizer reproduced real, confirmed races on the first attempt (2/2):

• bc250_CreateSurfaces / bc250_CreateBuffer vs bc250_DestroyBuffer (va_backend.c) — on the hwupload/DeriveImage path, plausibly the exact mechanism behind an intermittent SIGSEGV this release also narrowed (see below)
• gpu_compute_end_picture() vs gpu_compute_submitted_slot() (gpu_compute.c) racing the GPU double-buffer index directly

This is not specific to any one code path (it was found while investigating a CAVLC-only crash, but the racing functions are core VA-API entry points used regardless of entropy coder) — the default CABAC/production path shares the same unsynchronized calling contract. It has not caused a known failure in the confirmed-working v0.3.0 real-session use reported so far, but it is a real, live race, and a fix requires a lock spanning multiple files plus an audit of whether any locked path can block on vkWaitForFences(..., UINT64_MAX) while held (unaudited — a careless fix could trade a rare crash for an occasional hang). Full diagnosis, evidence, and recommended fix approach: docs/DEVLOG.md §26.1.2.

Fixed

CAVLC slice output buffer was undersized for CAVLC's real worst case, silently truncating the bitstream rather than crashing. The 768 bytes/MB allowance was correctly sized for CABAC but never updated when the CAVLC path's level_prefix≥15 escape coding (worst case ~2.3 KB/MB with int16-range coefficients) was added. Now sized per-coder. Only reachable via a client negotiating H.264 Baseline profile, or the BC250_USE_CABAC=0 test override — the default (CABAC/profile High) path is unaffected and verified byte-exact before/after.

quant_levels halved to int16_t (was int) — real values never approach the 32-bit range, verified device/driver support, byte-exact output. Also removed a GPU compute dispatch (entropy_encode.comp) that had been running unconditionally every frame with nothing downstream reading its output, wasting ~0.34 ms/frame.

Corrected guidance (things stated with confidence in earlier work that didn't hold up)

• A prior claim that CABAC lacked slice-boundary neighbor-context gating was wrong when it was written. A fresh audit of every CABAC context-derivation site found the gating already present, dating to the original CABAC commit — three days before the incorrect claim. No fix was needed; CABAC entropy coding is not blocked from multi-slice threading by this (a different investigation for a different day).
• Byte-exactness as a correctness oracle is narrower than previously documented. It is valid only for all-intra testsrc — plain testsrc with P-frames was shown non-deterministic too (same unmodified binary, 4 runs, 4 different outputs), not just moving content (testsrc2) as previously stated.
• A widely-repeated "real game drops 1440p streaming from 60 to 11 fps" figure has no real source and coincides with a number an earlier release had already retracted as a debug-logging artifact. A real, reproducible contention figure exists (~45× slowdown under a synthetic heavy GPU compute load), but no trustworthy real-game number does yet.

Investigated and explicitly not adopted (measured, not guessed)

Several plausible-looking performance levers were built, measured, and reverted because the numbers didn't support them — recorded in docs/DEVLOG.md §24–§26 so they aren't re-attempted from the same premise:

• Async compute queues: successfully enabled (no kernel changes needed, contrary to the community patch's own claim), verified correct, but useless here — this encoder's GPU stages are a strict dependency chain and the GPU is only ~34% of the frame.
• CPU/GPU frame pipelining: implemented, measured +2–3% (inside the noise floor) for a ~6 dB quality loss and a SIGSEGV under GPU contention. The overlap it was designed to create never actually happened.
• Sparse (mask-guided) coefficient transport: reverted — CAVLC/CABAC must code every 4×4 block in a coded group regardless of its content, so there was no valid bytes to skip, and the existing sequential copy was already near memory-bandwidth limits.
• Cross-slice entropy-coding threading: works (2.9× on the entropy stage, byte-identical output) but only on CAVLC, which costs ~24% more bitrate to reach — a bad trade. CABAC is where the real 9.4 ms is.

Known limitations (carried over, still true)

• Quality is bounded by the qp_min=12 floor rather than bandwidth at typical bitrates.
• In-loop deblocking is luma-only while the bitstream signals disable_deblocking_filter_idc=0; I-slice intra prediction reads source rather than reconstructed neighbours. Together ~3.7 dB of per-GOP drift; not visually significant.
• H.265/HEVC remains a non-functional stub — do not enable it.
• No trustworthy measurement yet of real-world behavior under actual game GPU contention (see "Corrected guidance" above).