Clarify NVENC live view startup failures - #19
Merged
robertjamesprior merged 2 commits intoSep 4, 2026
Conversation
robertjamesprior
marked this pull request as ready for review
September 3, 2026 15:20
sjmiller609
requested changes
Sep 4, 2026
sjmiller609
left a comment
There was a problem hiding this comment.
The error returned could currently return for a few different reasons. It would be improved if we can surface unambiguous errors, so that we don't show the user an out-of-memory error for a potentially different problem.
Please merge this PR in, then adjust related PRs if needed to handle the more specific errors. Thank you #20
## summary - probe the CUDA driver immediately when GStreamer cannot create `nvh264enc` - distinguish GPU memory exhaustion, missing GPUs, unavailable driver symbols, and non-CUDA NVENC/plugin failures - load `libcuda.so.1` at runtime so non-GPU images retain no CUDA build dependency, while keeping initialized CUDA process state mapped safely - release the global pipeline lock before running the diagnostic so a slow CUDA driver cannot stall existing streams or unrelated pipelines ## testing - `go test ./...` - `go test -race ./pkg/gst -count=1` - `go build -o /tmp/neko ./cmd/neko` - validated dynamic probing against fake CUDA drivers returning success, `CUDA_ERROR_OUT_OF_MEMORY`, and `CUDA_ERROR_NO_DEVICE` - validated on a real GPU browser: healthy CUDA and NVENC pipeline creation passed; forced GPU-memory exhaustion produced GStreamer's missing-`nvh264enc` error and the detailed `CUDA_ERROR_OUT_OF_MEMORY (2)` diagnosis; the slow-probe lock regression passed --------- Co-authored-by: sjmiller609 <7516283+sjmiller609@users.noreply.github.com>
sjmiller609
approved these changes
Sep 4, 2026
robertjamesprior
deleted the
hypeship/kernel-2101-neko-nvenc-error-main
branch
September 4, 2026 18:41
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CreatePipelinereturns the raw GStreamer message when pipeline construction fails. On GPU builds, a failed NVENC/CUDA init surfaces asno element "nvh264enc", which reads like a missing plugin even though the encoder is installed and the real limit is GPU memory.annotatePipelineErrorappends a short hint when all three hold:nvh264encnvh264encEvery other pipeline error passes through unchanged.
Test
go test ./pkg/gst -run TestAnnotatePipelineError -count=1The new test covers both missing-element wordings and one unrelated encoder error. Building the full
pkg/gstpackage needs the GStreamer dev headers, so run it where they are installed.Note
Low Risk
Changes affect error messaging and lock ordering on failed pipeline creation only; successful capture paths are unchanged aside from linking
-ldlfor the CUDA probe.Overview
When GPU live-view pipeline creation fails with a missing
nvh264encelement/plugin error,CreatePipelinenow appends a diagnostic hint instead of returning only the raw GStreamer message. A newgstreamer_cuda_context_probe(dynamiclibcudaload, init, device count, create/destroy context) reports whether CUDA is missing, misconfigured, out of memory, or healthy—so operators can tell GPU/CUDA limits from a truly absent plugin.annotatePipelineErroronly runs when the pipeline string and error both referencenvh264encand match known “no element” wordings; all other errors are unchanged. On the failure path,pipelinesLockis released before the probe so a slow CUDA check cannot block other pipelines.Tests cover NVENC error detection, lock release during probing, and the various
nvencFailureDetailmessage branches.Reviewed by Cursor Bugbot for commit 7ae62fc. Bugbot is set up for automated code reviews on this repo. Configure here.