Repository navigation
Releases: Shalasere/bc250-vulkan-encode-stopgap
Release list
v0.5.0
v0.4.0
v0.4.0 - HEVC P-frame quality fixed and board-validated (+19.6 dB), f…
v0.3.2 - CAVLC buffer fix, int16_t compaction, and a critical thread-safety finding
Perf/correctness follow-up to v0.3.1, plus a significant finding that changes the risk picture for this driver: it has no thread synchronization anywhere, and it is exposed to that under normal use, not just under an unusual test path.
⚠️ Known critical issue: no thread synchronization, TSan-confirmed data race (unfixed)
grep -rn 'pthread_mutex\|pthread_rwlock\|atomic_' src/ returns nothing. ffmpeg calls into this driver from ≥2 of its own concurrent threads (encoder_thread/filter_thread), and ThreadSanitizer reproduced real, confirmed races on the first attempt (2/2):
bc250_CreateSurfaces/bc250_CreateBuffervsbc250_DestroyBuffer(va_backend.c) — on the hwupload/DeriveImagepath, plausibly the exact mechanism behind an intermittent SIGSEGV this release also narrowed (see below)gpu_compute_end_picture()vsgpu_compute_submitted_slot()(gpu_compute.c) racing the GPU double-buffer index directly
This is not specific to any one code path (it was found while investigating a CAVLC-only crash, but the racing functions are core VA-API entry points used regardless of entropy coder) — the default CABAC/production path shares the same unsynchronized calling contract. It has not caused a known failure in the confirmed-working v0.3.0 real-session use reported so far, but it is a real, live race, and a fix requires a lock spanning multiple files plus an audit of whether any locked path can block on vkWaitForFences(..., UINT64_MAX) while held (unaudited — a careless fix could trade a rare crash for an occasional hang). Full diagnosis, evidence, and recommended fix approach: docs/DEVLOG.md §26.1.2.
Fixed
CAVLC slice output buffer was undersized for CAVLC's real worst case, silently truncating the bitstream rather than crashing. The 768 bytes/MB allowance was correctly sized for CABAC but never updated when the CAVLC path's level_prefix≥15 escape coding (worst case ~2.3 KB/MB with int16-range coefficients) was added. Now sized per-coder. Only reachable via a client negotiating H.264 Baseline profile, or the BC250_USE_CABAC=0 test override — the default (CABAC/profile High) path is unaffected and verified byte-exact before/after.
quant_levels halved to int16_t (was int) — real values never approach the 32-bit range, verified device/driver support, byte-exact output. Also removed a GPU compute dispatch (entropy_encode.comp) that had been running unconditionally every frame with nothing downstream reading its output, wasting ~0.34 ms/frame.
Corrected guidance (things stated with confidence in earlier work that didn't hold up)
- A prior claim that CABAC lacked slice-boundary neighbor-context gating was wrong when it was written. A fresh audit of every CABAC context-derivation site found the gating already present, dating to the original CABAC commit — three days before the incorrect claim. No fix was needed; CABAC entropy coding is not blocked from multi-slice threading by this (a different investigation for a different day).
- Byte-exactness as a correctness oracle is narrower than previously documented. It is valid only for all-intra
testsrc— plaintestsrcwith P-frames was shown non-deterministic too (same unmodified binary, 4 runs, 4 different outputs), not just moving content (testsrc2) as previously stated. - A widely-repeated "real game drops 1440p streaming from 60 to 11 fps" figure has no real source and coincides with a number an earlier release had already retracted as a debug-logging artifact. A real, reproducible contention figure exists (~45× slowdown under a synthetic heavy GPU compute load), but no trustworthy real-game number does yet.
Investigated and explicitly not adopted (measured, not guessed)
Several plausible-looking performance levers were built, measured, and reverted because the numbers didn't support them — recorded in docs/DEVLOG.md §24–§26 so they aren't re-attempted from the same premise:
- Async compute queues: successfully enabled (no kernel changes needed, contrary to the community patch's own claim), verified correct, but useless here — this encoder's GPU stages are a strict dependency chain and the GPU is only ~34% of the frame.
- CPU/GPU frame pipelining: implemented, measured +2–3% (inside the noise floor) for a ~6 dB quality loss and a SIGSEGV under GPU contention. The overlap it was designed to create never actually happened.
- Sparse (mask-guided) coefficient transport: reverted — CAVLC/CABAC must code every 4×4 block in a coded group regardless of its content, so there was no valid bytes to skip, and the existing sequential copy was already near memory-bandwidth limits.
- Cross-slice entropy-coding threading: works (2.9× on the entropy stage, byte-identical output) but only on CAVLC, which costs ~24% more bitrate to reach — a bad trade. CABAC is where the real 9.4 ms is.
Known limitations (carried over, still true)
- Quality is bounded by the
qp_min=12floor rather than bandwidth at typical bitrates. - In-loop deblocking is luma-only while the bitstream signals
disable_deblocking_filter_idc=0; I-slice intra prediction reads source rather than reconstructed neighbours. Together ~3.7 dB of per-GOP drift; not visually significant. - H.265/HEVC remains a non-functional stub — do not enable it.
- No trustworthy measurement yet of real-world behavior under actual game GPU contention (see "Corrected guidance" above).
v0.3.1
v0.3.1 - crash fix + 12.5% faster encode on moving content FIX A CRASH PRESENT IN v0.3.0. bc250_gpu_init() eagerly allocated every encoding buffer for 3840x2160 (~440 MB per encoder context) even at 1080p, and this board exposes only a 512 MB VRAM heap - which RADV also reports for the HOST_VISIBLE memory types the readback staging buffers use, so it is shared with the display. Sunshine's encoder probe creates up to 20 contexts, so allocations failed; every one of the 20 allocation results was ignored, so the failure surfaced as a SEGV in bc250_gpu_init instead of a clean "encoder unavailable" fallback. v0.3.0 was already logging 9 VK_ERROR_OUT_OF_DEVICE_MEMORY failures per Sunshine start and surviving only because the allocations that happened to fail were ones nothing later dereferenced. Buffers are now allocated lazily at the real resolution, and all 20 allocations are checked. Per Sunshine start: OOM 18 -> 0, SEGV 1 -> 0, gpu_init calls 20 -> 5. If Sunshine died at startup with status=11/SEGV in bc250_gpu_init on v0.3.0, upgrade. PERFORMANCE. quantize.comp already computed a per-4x4-block nonzero count into a buffer that had no host staging and no CPU consumer - computed every frame and never read. It now emits a bitmask instead, staged to the host at 1/16th the size of the coefficient buffer, and every "is this block all zero" question on the CPU (cbp_luma, cbp_chroma, the P_Skip decision, the CAVLC per-block path) is a bit test rather than a walk through 22 MB of mostly-zero data. 89.8-95.8% of blocks are entirely zero. 1440p moving content 45.7 -> 51.4 fps (+12.5%) 1440p static content 64.9 -> 71.4 fps (+10.0%) CAVLC time -17 to -21% at every resolution tested Quality unchanged: 58.44/58.52 dB PSNR, SSIM 0.9993, both PASS on the project's own gate. Mask verified exact against a CPU recompute on 200/200 frames, both frame types, two content types. KNOWN CONSTRAINT NOW DOCUMENTED. This encoder is not bit-reproducible on moving content - three runs of an identical configuration produce three different (all valid) bitstreams, ~0.02% size spread. Byte-exactness is a valid verification oracle only on content that pins at qp_min. See docs/DEVLOG.md 19.6. Full account, including two hypotheses that were measured and disproven on the way: docs/DEVLOG.md 19.
v0.3.0 - Real-world quality: rate control fixed
Real-world quality release. The long-standing "corruption during on-screen motion" report — the one that made this driver unusable for actual Sunshine/Moonlight streaming — is fixed, and it was not where previous releases said it was.
The fix
CBR filler bytes were being fed back into the rate-control feedback loop.
maybe_append_filler() pads each frame with a spec-legal filler_data_rbsp() NAL up to the per-frame bit target, and both encode paths handed that padded size to rc_update_stats(). Because filler exists precisely to make the frame hit the target, reported bits always cancelled the bucket drain, so buffer_fullness could never fall. The large opening IDR pushed it to its clamp, error stayed positive permanently, the integral term wound to full range, and QP saturated at its 51 maximum for entire sessions — every frame coded at the worst quantization the encoder allows, then padded back up so the bitrate looked correct.
That is why raising the requested bitrate never helped: the extra bandwidth bought padding, not picture.
Measured on a real remote client (2560×1440, 31 Mbps requested, 74 s, 2490 frames):
| before | after | |
|---|---|---|
| QP | pinned 51 | avg 12.0 (at the floor, near-lossless) |
| frame bytes | constant 64,559 (padding) | 2,084 – 148,542 (real picture) |
| delivered | 28.2 Mbps, mostly filler | 18.7 Mbps of real content |
Also fixed
- Rate-control bucket now drains by wall-clock time rather than a fixed per-frame quota. The fixed quota silently enforced the negotiated frame rate: a 1440p session negotiates 60 fps while this encoder sustains ~40, making the budget a third too small — measured 20.91 Mbps against a 30.99 Mbps request, with bits/frame matching the target to 0.01%.
target_percentageapplied on the sequence-parameter bitrate path, order-independently. Affected ffmpeg's-b:vCLI default (2× RC target); not exercised by Sunshine, which sendstarget_percentage=100.BC250_DEBUG_RC=1diagnostic added, andBC250_PERF_FRAMEnow logsqp/meas_fps/target_bpfbesidebytes. This is what exposed the bug.
Corrected guidance
Previous releases attributed the motion corruption upstream of this driver, to Sunshine's KMS capture or the compositor, on the evidence that pre-encode input and post-encode reconstruction were pixel-identical during a corrupted moment. That evidence came from a loopback test harness whose Moonlight client rendered onto the very display Sunshine was capturing — a video feedback loop compounding 4:2:0 loss every round trip. The conclusion was wrong. docs/DEVLOG.md §14–§16 records the five hypotheses investigated and discarded (chroma QP, the 10-bit capture path, a GPU-sync gap, luma-only deblocking, bitrate plumbing), four of which were argued from how the corruption looked, and all of which were wrong.
Newly documented prerequisite
Disable screen blanking on the host. Sunshine re-initializes KMS capture on every app launch; if the display has slept it reads the output as 0x0 and returns "Failed to initialize video capture/encoding" (Moonlight Error 503) to the client — even though Sunshine itself started cleanly hours earlier. This is the most likely reason a correct install appears broken. Verify with cat /sys/class/drm/card*-DP-1/enabled (kscreen-doctor -o is not reliable for this).
Known limitations
- Quality is now bounded by the
qp_min=12floor rather than bandwidth (~19 of 31 Mbps used at 1440p) — the next quality lever. rc_estimate_base_qp()saturates atqp_minabove roughly 31 Mbps at 1440p30.- In-loop deblocking is luma-only while the bitstream signals
disable_deblocking_filter_idc=0; I-slice intra prediction reads source rather than reconstructed neighbours. Together ~3.7 dB of per-GOP drift; not visually significant. - H.265/HEVC remains a non-functional stub — do not enable it.
v0.2.1
Contents
bc250_drv_video.so and its matching compiled shaders (shaders/*.spv), built from the current main.
Install: copy bc250_drv_video.so and shaders/ into the repository root, then run sudo ./tools/setup_bazzite.sh (Bazzite/Silverblue/Kinoite) or sudo ./tools/setup_steamos.sh (SteamOS/HoloISO). Both scripts require the compiled shaders directory; bc250_drv_video.so alone is non-functional.
Fixes since v0.2.0 (93 commits)
1. pic_init_qp_minus26 range violation.
h264_encoder_set_qp() wrote the raw QP into pps.pic_init_qp without the -26 offset the field's encoding requires (ITU-T H.264, bitstream.h). At QP ≈ 26, this produced a value one past the legal maximum (25), which any spec-compliant decoder rejects. Fix: pps.pic_init_qp = qp - 26. Not exercised by synthetic test content, which never sets a nonzero pic_init_qp.
2. P-slice skip decision omitted chroma residual.
The skip-legality check (mb_has_any_luma_nonzero) considered only luma coefficients. Per ITU-T 8.4.1.1, a P_Skip macroblock must carry zero residual for the entire macroblock, chroma included. A macroblock with nonzero chroma residual and zero luma residual was still certified skip-legal, dropping the chroma correction from the bitstream. Because the GPU-side reference reconstruction (reconstruct.comp) applied that same chroma residual unconditionally, the encoder's own future-frame reference retained a correction that was never transmitted — a one-way divergence uncorrected until the next IDR. Fix: mb_has_any_chroma_nonzero(), applied at both skip-decision sites (CAVLC, CABAC). Effect measured via the project's PSNR/SSIM harness: 51.5 dB → 59.6 dB average, SSIM 0.996 → 0.9994.
3. Slice bitstream buffer undersized for real content.
rbsp_buf_size = (end_mb - start_mb) * 64 + 4096 — a 64 bytes/macroblock budget calibrated against synthetic test patterns with low per-block AC energy. Real content at low QP exceeds this; on overflow, both the CAVLC and CABAC writers detect the condition and stop writing (no memory-safety issue) but the resulting bitstream is truncated at that point, desyncing any downstream decoder. Fix: 768 bytes/macroblock. Verified against captured frames from a real session that reproduced the overflow: PSNR 22.3 dB → 52.1 dB, zero decode errors, zero overflow events (previously logged as CABAC slice buffer overflow).
4. Executable bit lost on every .sh script in the repository.
Tracked file mode was 100644 (non-executable) across the board — an artifact of this repository having been edited from an NTFS checkout, which does not represent the Unix executable bit. Fixed via git update-index --chmod=+x on every script; internal script-to-script invocations of install_dkms.sh changed from ./install_dkms.sh to bash ./install_dkms.sh as a defense against recurrence.
Testing on hardware
- Driver load.
LIBVA_DRIVER_NAME=bc250 vainfoshould list the driver (AMD BC-250 RDNA2 Compute VA-API Driver) and enumerate at minimum an H.264 Main/High profile withVAEntrypointEncSlice. A missing driver entry orvaInitializefailure indicates the.soorLIBVA_DRIVERS_PATHis not correctly installed — re-check the setup script's summary output. - Shader load.
journalctl --user-unit=<sunshine unit>(or the equivalent stderr capture for whatever VA-API client is used) should show no[bc250-gpu]shader-load errors on first encode dispatch. This is where a missing/mismatchedshaders/*.spvset surfaces if the install-time check was somehow bypassed. - Synthetic correctness gate. If build tools are available,
tools/quality_test.shruns a real encode of synthetic content through the installed pipeline and scores decoded output against ground truth via PSNR/SSIM (pass threshold: 25 dB average). This does not require a streaming client and takes under a minute. - Real client session. Configure Sunshine's Video Encoder to VA-API, connect a Moonlight client, and inspect
~/.config/sunshine/sunshine.logforCLIENT CONNECTEDwith no immediately following decode/codec errors on the client side.grep -i overflowagainst the driver's stderr (journal, if run under systemd) should return nothing. - Sustained-motion / scene-change check. To exercise the open rate-control issue below: stream content with an abrupt large-area screen change (e.g. scrolling a content-heavy list, switching applications) and watch for a visible quality drop that persists for longer than roughly one GOP interval (default: a few seconds) rather than recovering within a frame or two. Its presence is expected pending a rate-control fix; its absence would itself be worth reporting, as it would suggest the underlying condition is content- or path-dependent in a way not yet characterized.
Open issue, not fixed in this release
Rate control (rate_control.c) responds to a large single-frame bitrate excursion by holding QP elevated for the duration required to drain a 1-second leaky-bucket buffer (RC_CBR mode), rather than the 2-frame low-latency buffer already implemented as RC_LOW_LATENCY and documented in-code as intended for this exact use case. rc_init() currently hardcodes RC_CBR regardless of context. Effect: a legitimate content-complexity spike produces a visible, sustained quality drop lasting up to a full GOP. Root cause identified and reproduced offline; no fix applied. See docs/DEVLOG.md for the full trace.