Skip to content

fix: robustness hardening — 14 confirmed crash/corruption defects - #22

Merged
jcelerier merged 39 commits into
mainfrom
fix/robustness-hardening
Jul 28, 2026
Merged

fix: robustness hardening — 14 confirmed crash/corruption defects#22
jcelerier merged 39 commits into
mainfrom
fix/robustness-hardening

Conversation

@jcelerier

Copy link
Copy Markdown
Owner

Fixes every library defect confirmed by the new lrd-hardening suite: each was empirically reproduced (signal, or measured corruption) before being fixed, and each fix is verified by the test that previously failed.

One commit per fix, in dependency order.

Commit Defect Was
03ef6bd L-01 null engine in the LRU cache SIGSEGV on any eviction after a failed load — including simply loading under a cap of 2. The path also stayed permanently unloadable after the file was restored.
855a06c L-02 inference before prepare SIGSEGV, all four orders
e0e8c1c L-11 handle validation / create out-param / stream drain double-destroy → SIGABRT (config) and SIGSEGV (pipeline); a destroyed handle answered queries with garbage (width 1771711538, synchronize()==SUCCESS)
55592e6 L-06 text dims vs config prepare_embeds(…,77,768) on a 1024-wide pipeline returned SUCCESS; OOB surfaced later as an opaque CUB error after partially writing the output
f51a05e L-16 schedule vs denoising_steps denoising_steps=2 with a 1-entry schedule returned SUCCESS while indexing past the end
0af4fa3 L-09 cache_interval / cache_maxframes interval=0SIGFPE; maxframes=-1 → unbounded VRAM growth
bdf1b1f L-14 non-finite scheduler coefficients NaN/inf/alpha=0 accepted; frames returned SUCCESS
97cec38 L-08 rgba_resize was dead code an unconditional return; meant it had never resized: a 256×256 input on a 512 bundle produced a pure black frame, byte-identical for two different inputs
fd7c92e L-03 img2img-turbo buffer sizes SIGSEGV on a guard page in all three directions
229327d L-05 RIFE output capacity measured 98 304 bytes written past a 32 768-byte buffer
d403c10 L-13 config_create initialised CUDA created a primary context on the default device before config_set_device was read, and broke a later fork()
7b824f3 L-15 silent feature drop a CN/IP the bundle can't honour was silently ignored; frames statistically identical to the plain model
6cb109c L-07 geometry validation config_set_dimensions accepted 185600² and 513×511
f230777 L-07 bonus: checked cudaMalloc 60 unchecked call sites; CUDATensor::data_ wasn't even in the member-init list

Verification

Full suite 31/31 green (was 20 expected failures). Behaviour regressions re-checked verbatim:

  • lrd_sched_change_testMAD(B_live, B_ref) = 0.00, OK
  • lrd_addnoise_testMAD(on,off) = 12.89, OK
  • graph_cn_ip_test cn → tracks at 99.0 dB, not baked (16.4 dB), CORRECT, graph speedup retained (1.07×)
  • L-08: at native resolution, frames are byte-identical (same FNV hashes) before and after re-enabling the resize.

Three things to read before merging

  1. L-03/L-05 are additive, not an ABI break. New *_sized entry points + capacity helpers (img2img_turbo_frame_bytes, rife_required_out_bytes), registered as optional symbols; the old 4-arg forms are marked deprecated. They still overflow if a caller lies about the buffer size — they cannot do better, since they are never told it. The score node should migrate to _sized; the loader plumbing is in place.
  2. -ffast-math makes std::isfinite() a no-op here. The L-14 guard inspects IEEE-754 bits instead. Relevant to any future validation code in this repo.
  3. L-16 rejects rather than truncating, and config_set_dimensions now enforces latent == image/8. Both are intended behaviour changes; every in-tree caller already complies.

Tests live in a separate local lrd-hardening repo (they need the engine bundles on this workstation).

🤖 Generated with Claude Code

jcelerier and others added 14 commits July 24, 2026 19:45
`TensorRTEngineCache::get_or_load()` called the loader and passed the result
straight to `insert()` without checking it. `loadEngineFromFile()` returns
nullptr on every failure it knows about — missing file, unreadable file, a
corrupt blob TensorRT refuses to deserialize, or not enough VRAM — so a single
failed load stored a null `EnginePtr` under that engine path.

Two consequences, both observed:

  * the path was poisoned against retry. Every later `get()` found the stored
    null, returned it, and the caller reported "failed to load" forever. The
    file could be put back and the same path still failed (-99) while the very
    same engine under its original path loaded fine.
  * the next eviction dereferenced it. `get_lru()` calls
    `entry.second->lastAccessTime()` on every entry, so both
    `engine_cache_set_max_entries()` and the loader's own make-room eviction
    turned into a SIGSEGV. Reproduced with no host cooperation at all: one bad
    path, then loading real engines under a cache cap of 2.

Fix: `insert()` refuses a null outright and `get_or_load()` returns the loader's
failure without recording it, so a failed load leaves the cache exactly as it
was and a retry can succeed. `get_lru()` additionally treats a null entry as the
oldest (evict it) rather than dereferencing it, as defence in depth.

Verified: capi/test_cache — all three cases now pass and their `[!shouldfail]`
tags are removed. `cache_count` after a failed load is 0 (was 1),
`set_max_entries(0)` and `engine_cache_clear()` survive, a repaired path reloads,
and a real load under a cap of 2 completes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
…duler (L-02)

The constructor brings up CUDA, the engines and the buffers and then reports
success, but it does not bring up the conditioning or the scheduler: those
arrive later, from the host, via prepare_embeds() and prepare_scheduler().
Nothing recorded that they had not arrived yet.

The denoise path dereferences both unconditionally — `prompt_embeds_->data()`
and `sub_timesteps_->data()` in every UNet forward, and
`beta_prod_t_sqrt_host_[0]` / `alpha_prod_t_sqrt_host_[0]` on the 1-step turbo
path, where an empty vector's data() is nullptr. All four orders (txt2img with
neither prepared, with embeds only, with the scheduler only, and img2img with
neither) SIGSEGV'd, which also made recovery unreachable: the process was dead
before the host could prepare properly.

Fix: `LibreDiffusionPipeline::inference_readiness()` reports what is missing
(conditioning, scheduler, or — for SDXL — the pooled embeddings / time ids that
every SDXL forward binds), and every inference entry point in the C API
(txt2img, img2img, their _gpu variants, txt2img_sd_turbo_gpu, predict_x0_batch)
returns LIBREDIFFUSION_ERROR_NOT_INITIALIZED with a message on stderr instead of
running. Preparing properly afterwards still produces correct frames.

Verified: capi/test_ordering — all five cases now pass (four premature-call
cases plus the recovery case, which renders a real frame, std > 1, after the
rejected call); `[!shouldfail]` tags removed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
…n the stream (L-11)

Three separate lifetime holes at the C boundary, all confirmed:

  * DOUBLE-DESTROY. No handle carried a magic word, so the second delete freed
    an already-freed block: glibc "free(): double free detected" -> SIGABRT for
    a config, SIGSEGV for a live pipeline (the whole dtor chain — CUDA stream,
    graph, TensorRT contexts — re-run on freed memory).
  * USE-AFTER-DESTROY did not crash, it LIED. A destroyed config read back a
    garbage width (1771711538) and accepted set_batch_size with SUCCESS; a
    destroyed pipeline reported num_runtime_loras()==0 and
    pipeline_synchronize()==SUCCESS.
  * A FAILED pipeline_create left the caller's out-param untouched and leaked
    the wrapper (the throw escaped before `*pipeline = p`, and nothing owned p).
    Every entry point then dereferenced the caller's stale value inside its own
    `!pipeline` guard.

Fix: librediffusion_config_t / librediffusion_pipeline_t each carry a magic word
set on construction and CLEARED immediately before the block is freed; valid()
checks it and every entry point goes through valid()/valid_handle() instead of a
bare null test. config_create / config_clone / pipeline_create always write the
out-param (nullptr first, the handle only on success) and pipeline_create now
owns the wrapper through a unique_ptr so a throwing constructor frees it.

Also closes the destroy-racing-inference window: ~LibreDiffusionPipeline
synchronizes the stream before tearing anything down, and img2img's trailing
cudaStreamSynchronize — commented out, while its D2H into the CALLER's host
buffer is async — is restored. txt2img always synchronized; img2img did not.

Verified: capi/test_lifetime — the four L-11 cases now pass and their
`[!shouldfail]` tags are removed (double-destroy of a config and of a live
pipeline both survive; a destroyed config reports width 0 and rejects the
setter; a destroyed pipeline rejects synchronize; a failed create writes NULL).
L-13 in the same file still fails as expected, and is fixed separately.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
… (L-06)

prepare_embeds() sizes its device buffer from the CALL's arguments
(batch_size * seq_len * hidden_dim) while every consumer reads
config_.text_seq_len * config_.text_hidden_dim back out of it through a raw
cudaMemcpyAsync that bounds-checks nothing. Declaring a smaller shape is
therefore an out-of-bounds DEVICE read.

Confirmed: prepare_embeds(..., 77, 768) on a 1024-wide pipeline returned
SUCCESS. The damage surfaced one call later, somewhere else, as an opaque
"transform: failed inside CUB: cudaErrorInvalidDevice" (-99) — after the output
buffer had already been partially written (mean 171).

Fix: reject the mismatch at the seam where it is still cheap to reject.
check_text_dims() compares the declared [seq_len, hidden_dim] against the
pipeline's configuration and returns LIBREDIFFUSION_ERROR_INVALID_DIMENSIONS
with both shapes on stderr. Applied to prepare_embeds, prepare_null_embeds,
prepare_negative_embeds and blend_embeds.

Same class, same treatment for the neighbours the ledger names:
  * set_ipadapter_tokens rejects a `dim` that is not the pipeline's
    cross-attention width (the extended-ehs assembly reads
    ipadapter_num_tokens_ * text_hidden_dim out of a buffer sized num_tokens*dim),
    and rejects non-positive num_tokens/dim.
  * prepare_sdxl_conditioning passes NO shapes at all — both buffers are read
    using the pipeline's own dimensions — so it is refused outright on a
    pipeline with no SDXL conditioning configured, and librediffusion_c.h now
    documents the exact shapes the caller must supply.

Verified: capi/test_dims — "L-06 prepare_embeds with a smaller hidden dim must
be rejected" now passes (prepare_rc = -7 instead of 0) and its `[!shouldfail]`
tag is removed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
…; bound the loops (L-16)

The step count lived in two independent places and nothing kept them in
agreement: `config_.denoising_steps` (set on the config, and what
init_buffers()/reinit_buffers() size every batch buffer from) and the length of
the five coefficient vectors (set by prepare_scheduler). The denoise loops
iterated the former and indexed the latter.

Confirmed: denoising_steps=2 with a 1-entry schedule — both prepare_scheduler
and txt2img returned SUCCESS and produced a plausible frame (std 59) while
reading alpha_prod_t_sqrt_host_[1] past the end of a 1-element std::vector. The
earlier equal-span guard only compares the five arrays to EACH OTHER, which is
exactly where the remaining desync lived.

Fix, in three layers:

  1. prepare_scheduler refuses a length that disagrees with
     config_.denoising_steps (and an empty schedule outright), naming both
     numbers. This never fires on a legitimate live update: a genuine step-count
     change goes through reinit_buffers first — which sets denoising_steps AND
     reallocates — and only then pushes the matching schedule. It does fire on
     the score node's `continue`-on-out-of-range-index path, which silently
     produced a short schedule.
  2. every denoise loop is now bounded by `denoise_steps()` =
     min(config_.denoising_steps, schedule length) rather than by the declared
     count, so the loops stay in bounds even if the two ever drift again.
  3. the unchecked indexers are gone: alpha_at/beta_at/c_skip_at/c_out_at
     bounds-check and throw a message naming the array, the index and the
     schedule length. That covers add_noise (unet.cpp:105-114),
     scheduler_step_batch, the sequential multi-step path, the turbo
     txt2img [0] reads and add_noise_direct/encode_image in vae.cpp:13-14 —
     each of which was a raw operator[] whose empty-vector case is a null deref.

Verified: capi/test_scheduler — "prepare_scheduler with zero timesteps must be
rejected" and "fewer coefficients than denoising_steps must not read past the
end" both pass now and their `[!shouldfail]` tags are removed. The NaN/inf/alpha0
case in the same file still fails as expected and is fixed separately (L-14).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
`enable_temporal_coherence` / `config_set_temporal_params` took five numbers and
validated none of them. Two of them are not merely wrong when out of range, they
are fatal:

  * cache_interval is the divisor of `frame_id % config_.cache_interval` in
    img2img_impl. Zero is an integer-division trap — a SIGFPE that no try/catch
    can intercept and no host can survive. Confirmed: enable_temporal_coherence
    with cache_interval=0 returned SUCCESS and the next img2img died.
  * cache_maxframes is compared against a size_t in
    `while(cached.size() > (size_t)cache_maxframes)`, so -1 becomes
    18446744073709551615 and the deque never drops a frame: one latent of VRAM
    per frame, forever. Confirmed: -1 accepted with SUCCESS, 30 consecutive
    frames cached with zero evictions.

Both are exactly what an unclamped "update every N frames" / "keep N frames"
control produces at its lower bound.

Fix: both C-API setters reject < 1 with LIBREDIFFUSION_ERROR_INVALID_ARGUMENT;
enableTemporalCoherence() refuses them too so no internal caller can install
them; and reinit_buffers(), which takes a whole config from the host, clamps
them to 1 rather than copying them through.

Verified: capi/test_temporal — both L-09 cases now pass and their
`[!shouldfail]` tags are removed; the L-10 no-crash regression guard still
passes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
prepare_scheduler accepted NaN, inf and alpha = 0 and the following frame came
back with LIBREDIFFUSION_SUCCESS. NaN and inf both propagate through the whole
latent in one multiply and yield a pure-white frame (mean 255, std 0); alpha = 0
is a DIVISOR on the single-step turbo path (x_0_pred /= alpha) and yields
mean 249 / std 35. A host automating a coefficient through a bad value — a
crossfade, an interpolated preset, a divide that produced inf — was told
nothing.

Fix: all five coefficient arrays are checked for finiteness, and
alpha_prod_t_sqrt is additionally checked for zero, before anything is copied to
the device; the throw names the array and the index. Timestep VALUES stay
unconstrained beyond finiteness: -999 and 1e30 both produce ordinary frames, so
they are not the library's business.

Note the checks inspect the IEEE-754 bits rather than calling std::isfinite()
or comparing against 0.f. The library is compiled with -ffast-math, so the
compiler is entitled to assume no NaN/inf exists — exactly the assumption these
inputs violate. Measured: an isfinite()-based version let inf straight through
(and caught NaN only by accident, via `NaN == 0.f` folding to true).

Verified: capi/test_scheduler — "NaN / inf / zero coefficients must not silently
produce a frame" now passes (nan/inf/alpha0 all rejected at prepare_scheduler
with -99, the following inference correctly reports NOT_INITIALIZED, and the
two pathological-but-legal timestep probes still render ordinary frames);
`[!shouldfail]` removed. The whole file is green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
…pe not area (L-08)

`rgba_resize()` began with an unconditional `return;` immediately before the
nppiResize call — it had NEVER resized anything. Every img2img whose input pixel
count differed from the configured size therefore ran the VAE on the
UNINITIALISED VRAM of the staging buffer. Confirmed on a 512px bundle: a 256x256
frame returned SUCCESS and a pure black image, byte-identical for two completely
different inputs (mean 0, std 0), while the native resolution reacted normally
(std 60 / 73).

Compounding it, the "is this already the right size?" test compared AREA
(iw*ih == width*height), so 1024x256 was accepted as if it were 512x512 and
processed with the wrong strides — silently scrambled rather than resized.

Three changes:

  * the dead `return;` is gone, and a non-NPP_NO_ERROR status now throws instead
    of printing to stderr: a failed resize leaves the destination untouched,
    which is exactly the uninitialised-VRAM situation this function exists to
    prevent.
  * init_npp() no longer declares a LOCAL `NppStreamContext npp_stream_` that
    shadows the member of the same name. Every field was being written to the
    local, which was then discarded, leaving the member uninitialised — harmless
    only while the resize was dead, and an instant crash the moment it was not.
    The member is now zero-initialised, filled, and its nStreamFlags taken from
    cudaStreamGetFlags.
  * both size tests compare SHAPE.

Verified on the GPU, capi/test_dims "L-08 img2img at a resolution the pipeline
was not built for": with a 512px bundle, 256x256 now yields two DIFFERENT
outputs for two different inputs (hashes 8796399020846582099 vs
7324705845932008229, mean 129.3/96.3, std 60.2/73.3 — i.e. the same statistics
as the native-resolution frames instead of all-zero), and so does 1024x256. The
native-resolution path is byte-for-byte unchanged (same hashes as before the
patch). `[!shouldfail]` removed; the file's only remaining expected failure is
the L-07 config seam, fixed separately.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
`librediffusion_img2img_turbo_frame(h, in_rgba, ehs, out_rgba)` takes three host
buffers and no length for any of them. The implementation copies H_*W_*4 bytes
OUT of in_rgba and INTO out_rgba — where H_/W_ are the ENGINE's static 512x512,
not anything the caller said — plus a flat 1*77*1024 floats (315 392 bytes) out
of ehs. A caller working at any other resolution, or with a different text
width, overflows by construction. Confirmed with guard pages: SIGSEGV in all
three directions (256x256 output, 256x256 input, and a 77x768 SD1.5-width
embedding).

Fix: new entry points that carry the declarations —

  librediffusion_img2img_turbo_frame_sized(h, in, in_bytes, ehs, ehs_elements,
                                           out, out_bytes)
  librediffusion_img2img_turbo_frame_dev_sized(h, in, in_bytes, ehs_dev,
                                               out, out_bytes)

plus librediffusion_img2img_turbo_frame_bytes() / _ehs_elements() so a caller can
ask what the loaded engines actually require. A mismatch is
LIBREDIFFUSION_ERROR_INVALID_DIMENSIONS with both sizes on stderr.

This is ADDITIVE, not an ABI break: the existing 4-argument entry points keep
their signatures (the score node calls them through dlsym'd pointers) and are now
thin forwards that pass the model's own sizes — i.e. exactly their historical
behaviour, which by construction cannot detect a caller whose buffers are
smaller, because it is never told how big they are. They are marked DEPRECATED in
librediffusion_c.h and the new symbols are registered as OPTIONAL in
librediffusion_loader.hpp so the node can migrate without a hard version bump.

Verified: capi/test_i2it — the three L-03 overflow cases now drive the sized
entry point, are refused with -7 instead of trapping on the guard page, and their
`[!shouldfail]` tags are removed. The baseline case additionally checks that
frame_bytes/ehs_elements report 1048576/78848 and that the sized call renders a
frame identical to the unsized one. 5/5 green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
`librediffusion_rife_interpolate` writes 2^exp * H*W*4 bytes into a caller
buffer that carries no length, and the frame COUNT comes from state the caller
set in a different call (`set_interpolation_exp`). The two drift apart
trivially: any host that raises the interpolation factor without re-sizing its
output buffer overflows it, and nothing in the API can notice. Measured with a
canary: 98 304 bytes written past a 32 768-byte buffer when exp went 1 -> 3.

`set_interpolation_exp` also clamped only from below, storing anything it was
given — a readback of 2147483647 — while the internal `total_out *= 2` loop runs
`exp` times in a signed int (UB from exp = 31 up).

Fix, both halves:

  * librediffusion_rife_interpolate_sized() / _gpu_sized() take
    out_capacity_bytes and return LIBREDIFFUSION_ERROR_INVALID_DIMENSIONS,
    naming the shortfall, when it is smaller than the 2^exp sequence they are
    about to write. librediffusion_rife_required_out_bytes(h, H, W) reports the
    requirement for the current exp/enabled state so a host can size correctly
    in the first place.
  * set_interpolation_exp clamps to [0, LIBREDIFFUSION_RIFE_MAX_EXP] (= 4, i.e.
    16 frames per real frame — already far past what any display pipeline
    consumes), so the count arithmetic can no longer overflow.

Additive, like L-03: the existing unsized entry points keep their signatures for
the score node's dlsym'd pointers and are marked DEPRECATED in the header; the
new symbols are registered as OPTIONAL in librediffusion_loader.hpp.

Verified: capi/test_rife, 7/7 green, both `[!shouldfail]` tags removed.
"a buffer sized for exp=1 must not be overflowed at exp=3" now drives the sized
form and gets -7 instead of a guard-page trap; "a huge exp must be rejected"
reads back 4 for 31/63/INT_MAX instead of the value verbatim; and the canary
case now asserts BOTH that the deprecated entry point still writes 98 304 bytes
past the end (it cannot do better — it is never told the capacity) and that the
sized form refuses the identical call without writing a further byte.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
…process (L-13)

Every entry point wrapped in `try_catch_wrapper` ends with `check_cuda_error()`,
whose `cudaGetLastError()` BRINGS UP the CUDA primary context. The config entry
points — which allocate a struct of ints and strings and touch no device — were
wrapped in it too.

So `librediffusion_config_create()` cost a full context creation (hundreds of
ms, hundreds of MB of VRAM) on whatever thread happened to deserialise a preset,
pinned the context to the DEFAULT device before `config_set_device` was ever
read, and left the process unable to fork a working child. Measured: a child
forked after `version()` can cudaMalloc; a child forked after `config_create()`
gets cudaErrorInitializationError.

Fix: a `try_catch_host()` wrapper with identical exception handling and no CUDA
call, used by every `librediffusion_config_*` entry point (create, clone, the
engine-path and timestep-index setters, the SDXL/IP-Adapter/temporal setters).
`try_catch_wrapper` stays exactly where a CUDA call really may have happened —
every pipeline entry point.

Verified: capi/test_lifetime — "config_create must not initialize CUDA in the
calling process" now passes (a child forked after config_create can cudaMalloc
again) and its `[!shouldfail]` tag is removed. 7/7 green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
…(L-15)

A ControlNet or IP-Adapter configured against a UNet engine that cannot honour it
was SILENTLY DROPPED. init_engines printed a warning to stdout (ControlNet) or
said nothing at all (IP-Adapter), disabled the feature, and the pipeline then
rendered ordinary frames while the host believed its control image or style image
was steering the output. Measured: the frame was statistically identical to the
plain model's — std 52.0619 in both. The only hint a host ever got was
set_controlnet_cond_rgba returning -99 for an index that no longer existed; the
IP-Adapter path gave no hint whatsoever, since set_ipadapter_tokens returned
SUCCESS.

Fix: both are now configuration errors. init_engines throws — naming the engine
path and what it lacks — when ControlNets are configured against a UNet with no
input_control_* inputs, or when IP-Adapter was requested against a UNet that is
not an IP variant. pipeline_create returns the error and the host finds out
immediately instead of after shipping a take.

"Requested" needs recording for IP-Adapter, because ipadapter_num_tokens/scale
have usable defaults and so cannot distinguish "asked for" from "left alone":
LibreDiffusionConfig gains `ipadapter_requested`, set by
librediffusion_config_set_ipadapter. AUTO-detection is unchanged — an IP-variant
engine still enables itself when the host did not ask, which is what the
"IP-Adapter bundle driven WITHOUT IP-Adapter configured" case relies on.

Both score-node call sites are already workflow-gated (config_add_controlnet only
for ControlNet workflows, config_set_ipadapter only when ipadapter_enabled), so
this reports exactly the mismatch and never a normal workflow.

Verified: capi/test_mismatch — "ControlNet configured against a PLAIN bundle" and
"IP-Adapter configured against a PLAIN bundle" now pass, `[!shouldfail]` removed;
all 12 cases green, including the recovery half (a correct pipeline still comes up
and renders in the same process) and the whole previously-cleared wrong-bundle
matrix.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
… (L-07)

The setter rejected <= 0 and nothing else, so every impossible geometry was
accepted verbatim and only contained downstream by luck:

  * 185600x185600 with a 23200x23200 latent grid — batch*4*lh*lw overflows the
    int it is computed in, goes negative, becomes a huge size_t, and reaches a
    cudaMalloc whose result is never checked. What saved it was the sticky CUDA
    error that try_catch_wrapper's trailing cudaGetLastError() happened to
    report as -4.
  * 513x511 — the VAE's 8x downsampling cannot express it.
  * 512x512 with an 8x8 latent grid — every kernel indexes one grid and the
    engine the other; TensorRT's setInputShape happened to refuse the sample
    shape at the first inference, several hundred megabytes of allocation later.

Fix: validate where the caller still has a return code to look at. Positive,
<= 16384 (well past any diffusion model, and it keeps every width*height*4 and
batch*4*lh*lw product far inside an int), a multiple of 8, and a latent grid that
is exactly width/8 x height/8. Documented in librediffusion_c.h. Every in-tree
caller — the score node, validation_cpp, and all the upgrade-test harnesses —
already passes W/8, H/8.

The tier-3 fixture had to be corrected alongside: make_config() ignored the
setter's return code, so a refused geometry silently left the config at its
DEFAULT 512x512 and built a perfectly good pipeline, hiding the refusal. It now
treats a rejected geometry as "no config".

Verified: capi/test_dims — "config_set_dimensions rejects impossible geometry"
now passes and its `[!shouldfail]` tag is removed; the two downstream cases
(overflowing allocation, inconsistent latent grid) now fail at the setter instead
of deep inside CUDA/TensorRT. 6/6 green — the whole file is clear.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
…bonus)

`CUDATensor(size_t)` did not list `data_` in its member-init list and discarded
the cudaMalloc return code, so a failed allocation — an overflowed element count,
genuine VRAM exhaustion — left the object holding an INDETERMINATE pointer that
every later kernel launch and cudaMemcpy dereferenced. There are ~60 construction
sites and no CUDA_CHECK anywhere; what has been saving them is the sticky CUDA
error a subsequent call happens to report.

Fix, following the pattern RifeInterpolator::ensureScratch already establishes:
initialise data_ to nullptr, reject an element count whose byte size wraps
size_t, treat a zero-length tensor as owning nothing, and throw a message naming
the byte count and the CUDA error when cudaMalloc fails — so the C boundary turns
it into an error code instead of a device fault at an unrelated site.

Kept as its own commit because it touches every allocation in the library.

Verified: the full tier-3 C-API suite is green afterwards (12 binaries,
test_lifetime 22, test_cache 13, test_ordering 16, test_dims 19, test_temporal 8,
test_scheduler 11, test_i2it 20, test_rife 39, test_mismatch 60, test_resource 8,
test_sana skipped). test_resource's VRAM-exhaustion cases in particular still
behave: an unused cached engine is evicted so a load that would not otherwise fit
succeeds, and an unreclaimable shortfall reports -99 with the process still able
to load a smaller model.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
@gemini-code-assist

Copy link
Copy Markdown

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

jcelerier and others added 15 commits July 25, 2026 05:51
computeClipEmbeddings_SDXL asks both encoders for ONE prompt, so each source
holds a single [77, h] row, but the concat loop read them at (b*77+s) for b in
[0, batch_size). Every batch element past the first was a device over-read;
pooled_embeds was returned one row deep while prepare_sdxl_conditioning reads
batch_size * pooled_dim from it. SIGSEGV on 16 SDXL engines in the sweep.

Broadcast row 0 (all batch elements carry the same prompt), size the pooled
buffer by batch, take the per-token and pooled widths from the engines instead
of the hardcoded 768/1280/2048, reject batch_size < 1, and check the three
allocations.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
…all (F-03, F-04)

A failed allocation leaves cudaErrorMemoryAllocation pending in the runtime.
Every entry point ends in cudaGetLastError(), and the throwing paths never read
it, so the error was reported by the NEXT, entirely valid call: after a rejected
oversized create, pipeline_create and prepare_embeds returned -4 for a bundle
that works in a fresh process. Same mechanism on the klein path, where a failed
flux2_stream_create left the error for the next pipeline_create.

Drain the pending error in every catch and in the entry points that hold a
cudaError_t of their own, so the failure is attributed to the call that caused
it. Class handled: the CLEARABLE per-call errors.

The unclearable kind is separated rather than papered over. An illegal address,
launch failure or ECC error kills the context; cudaGetLastError cannot clear it
and every later call returns it forever. Those are classified, latched, and
reported as the new LIBREDIFFUSION_ERROR_CUDA_CONTEXT_LOST (-9) so a host sees
"the context is dead, restart" instead of a misleading -4 on unrelated work.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
Th=Tw=0 returned a VALID handle: flux2_stream_dims reported 0x0, every device
buffer was a zero-byte allocation, and the context was unusable afterwards. A
grid with no tokens cannot produce an image, so fail the create.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
… buffers (F-02)

img2img/txt2img take ONE host RGBA frame and no length; img_preprocess and
img_postprocess convert exactly one image. Both host<->device copies were
nevertheless sized batch_size * iw*ih*4, so a batch-2 pipeline read a frame past
the end of the caller's input and wrote a frame past the end of its output. That
is a heap overflow of memory this library does not own, which is why the sweep
saw it surface as SIGSEGV inside cudaFree at teardown (the clobbered heap held a
CUDATensor whose device pointer was overwritten) or as glibc heap corruption,
after inference had already returned successfully. The device staging buffers
keep the batch extent; only the host copies are corrected.

Two device-side over-reads on the same trigger, both found with compute-sanitizer
("Copy is larger than memobj size, for source operand") and both gone now:

  - sub_timesteps_ held denoising_steps entries but every forward copies
    total_batch (or batch_size) floats out of it. It now holds the schedule
    cycled to that extent, so the first n rows stay bit-identical and the surplus
    repeats the schedule; inference_readiness reports a schedule prepared for a
    smaller batch instead of reading past the end.
  - stock_noise_/init_noise_ were sized by denoising_steps but read at the latent
    extent by cfg-self/cfg-initialize and add_noise.

All three coincide only when batch_size == denoising_steps * frame_buffer_size,
which is why nothing before batch 2 hit them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
…F-06, F-07)

L-15 established the rule for ControlNet and IP-Adapter; four more places kept
returning SUCCESS and rendering a frame byte-identical to the one the host would
have got without asking for anything.

  - enable_temporal_coherence on an engine with neither kvo_cache_in_* nor
    attention_* outputs: printed to stderr, disabled itself, returned 0. The
    parameter checks stay first so L-09's diagnostics do not change.
  - MODE_TEMPORAL_V2V declared at create against the same: same treatment, now
    an error at init_engines beside the ControlNet / IP-Adapter checks.
  - set_lora_scale / set_lora_scale_vector on a slot the engine does not
    declare: UNetWrapper bounds-checked and returned silently, so a bundle with
    zero runtime LoRA slots accepted every scale.
  - SDXL added conditioning against a UNet with no text_embeds input: both
    config_set_sdxl_config and prepare_sdxl_conditioning returned 0 and the
    buffers went nowhere. UNetWrapper now detects the input; the inverse (an
    SDXL engine driven as SD) is refused too, since the SD path never binds it.
  - model_type MODEL_FLUX2_KLEIN_4B through the SD C API: "recorded for
    completeness" meant a mis-declared host got ordinary SD frames and no
    signal. klein has its own entry points.

NOT fixed here: flux2_stream_set_steps(-1). It returns void, so reporting needs a
signature change, and that breaks tests/mock/mock_api.cpp in the hardening repo,
which this change may not touch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
…-06)

librediffusion_flux2_stream_set_steps ignored num_steps <= 0 and had no way to
say so. Returns librediffusion_error_t now; NULL handle and non-positive counts
are reported. Callers that ignore the result are unaffected.

Completes the F-06 row that was blocked on the mock's matching signature.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
…block (S-03)

L-11 gave each handle a magic word set on construction and cleared just before
the delete. The word lives INSIDE the block it is meant to validate, so every
call the API now documents as safe — double-destroy, and any query after destroy
— read freed memory to decide the block was freed. That is a heap-use-after-free
in the guard itself, and it is the guard for the one operation hosts are told
they may perform.

Invisible against the shipped .so; the ASan-instrumented build reported it four
times (config_destroy, config_get_width x2, num_runtime_loras).

It also does not work. glibc's tcache happens to overwrite the first 16 bytes of
a freed chunk with values that do not equal the magic, which is the whole reason
the scheme appeared to hold. Hand the block back out to an allocation that
contains the byte pattern and the stale handle re-validates as live: measured,
config_get_width on a destroyed handle returns 1280459843 (0x4C524443, 'LRDC'),
config_set_batch_size returns SUCCESS and writes into a buffer that now belongs
to someone else, and the following destroy segfaults.

Replaced with a registry of live handles, consulted by pointer value, which
never dereferences an untrusted pointer. erase() decides the race, so two
concurrent destroys cannot both free. The tables are deliberately leaked: a host
may destroy a handle from a static destructor of its own, after any
function-local static would already have been torn down. Cost is a mutex and a
hash lookup on calls that already do far more.

Only config and pipeline carried the magic scheme, so only those two change; the
clip / flux2 / flux2_stream / rife / img2img_turbo handles have never had a
double-destroy guard at all, which is a separate gap.

test_lifetime, instrumented .so: 4 reports / rc=2 / 2 cases failed -> 0 reports /
rc=0 / 35 assertions. All 16 tier-3 binaries green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
…heduler (S-01)

compute-sanitizer initcheck, cs_probe batch-frames (V2V bundle, batch_size=2, one
img2img): 787 456 uninitialised device reads. Two distinct defects, both the
"batch not handled" family, both invisible in-process — nothing is out of bounds,
the buffers are simply unwritten, so memcheck is clean on the same case.

786 432 of them (512x512x3, exactly one frame of RGB) in
rgba_to_rgb_normalized_fp32_kernel. img2img uploads ONE host frame (the F-02 fix,
correct: the caller's buffer holds one) and rgba_resize likewise writes one, but
img_preprocess converts config_.batch_size frames out of that staging buffer and
the VAE encoder runs batch_size rows.

Fixed by replicating the supplied frame across the batch extent rather than
shrinking the work to one row. The whole latent geometry — vae_encoded_x_t_latent_,
the predict_x0_batch buffers, unet_batch_size, the embedding tiling — is sized from
batch_size in init_buffers(), so encoding a single row would desynchronise all of
it. Replication is also the only defensible content: on this machine the one UNet
profile that admits batch 2 is the V2V bundle, whose extended attention concatenates
K/V across the batch, so row 0's own output depends on what the other rows contain.

The remaining 1 024 were a second defect the first one had been masking:
scheduler_step_batch ignored its element count and passed a hardcoded batch of 1 to
launch_scheduler_step_fp16 ("FIXME double-check that batch size is 1"). At
batch_size > 1 it wrote only the first row of predict_x0_batch_denoised, and the
store_d2d that follows copies latent_size — so every row but the first was read
unwritten, and would have been garbage rather than merely undefined. The count is
now derived from the N every caller already passes; the callers that step one row at
a time pass exactly one stride and are unaffected.

Severity was "silent and unreproducible" rather than "corrupt output today": a fresh
process gets zeroed pages from the driver, which is why three runs fingerprinted
identically. That does not survive a long-lived host, where the staging buffer is
regrown over VRAM something else has used.

initcheck 787 456 -> 0 on all seven cases. memcheck unchanged (l10 5, the open
L-10; every other case 0). batch-frames returns the same out_hash as before, and
lrd_sched_change_test / lrd_addnoise_test / graph_cn_ip_test report byte-identical
metrics — batch 1 divides to exactly the previous single row. All 16 tier-3 binaries
green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
pipeline_reinit_buffers took a whole config and installed it. It cannot reload an
engine, so a geometry outside the loaded engines' optimization profiles is
unusable — but it returned 0, and the refusal arrived at the next inference as
TensorRT's own

  setInputShape: ... Set dimensions are [1,3,1024,1024].
                     Expected dimensions are [-1,3,512,512]

and an opaque INTERNAL (-99) from img2img. F-07's class at a seam F-07's fix did
not cover: the call that could have refused reported success.

The new geometry is now checked against the profiles of all three engines before
anything is mutated — static dimensions must match, dynamic ones must fall inside
profile 0 — and reinit_buffers returns INVALID_DIMENSIONS with the dimension that
does not fit:

  [librediffusion] INVALID_DIMENSIONS: VAE encoder engine input 'images' is a
  fixed 512 in dimension 2, but [1, 3, 1024, 1024] asks for 1024

The UNet is checked at unet_batch_size(), the loosest batch it is ever driven at
(cfg-full doubles it), so the check cannot refuse a geometry that would have
worked. An absent tensor or an unrecognised rank constrains nothing.

CONSEQUENCE FOR L-10, which is NOT fixed here. The stale-temporal-buffer sequence
needs reinit_buffers to grow config_ past the buffers, and the only growth that
ever did so was one the engines could not honour. On every bundle on this machine
the VAE encoder is a fixed 512 and the decoder a fixed 64, so no resize is
admissible at all and L-10 is now unreachable rather than repaired: compute-
sanitizer memcheck cs_probe l10 goes 5 errors -> 0 for that reason alone. It stays
open, and comes back the moment a bundle ships with a genuine resize range.

All 16 tier-3 binaries green (374 assertions), all 20 no-GPU binaries green,
initcheck and memcheck 0 on all seven cases, and lrd_sched_change_test /
lrd_addnoise_test / graph_cn_ip_test unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
Comment-only. Removes the narration the hardening commits left behind —
defect IDs, before/after histories, measured symptoms, multi-line
justification of one-line changes — and keeps the short notes that
explain non-obvious mechanics (the -ffast-math NaN caveat, why
sub_timesteps_ is read by batch extent, why one frame is replicated
across the batch, why try_catch_host must not touch CUDA) plus the
public C-API contract docs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
…le ones

cudaErrorAssert (710) and cudaErrorNvlinkUncorrectable (220) are sticky per the
CUDA runtime: once raised, every later call in the context returns them. They
were absent from cuda_error_is_context_fatal(), so a device-side assert left the
library reporting a per-call error and retrying forever.

The reverse problem for cudaErrorContextIsDestroyed and cudaErrorDeviceUninitialized:
they are NOT sticky. A host that calls cudaDeviceReset() -- perfectly legal, and
outside our control in an embedding application -- gets one of them once, and
under the old list that write-once latch then refused every entry point including
pipeline_create for the rest of the process. Since g_cuda_context_lost has no
clear path by design (a genuinely lost context cannot be recovered by asking
nicely), the fix is to stop latching errors that are not actually terminal,
rather than to add a librediffusion_clear_cuda_context_lost() escape hatch whose
only correct use would be undoing this misclassification.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
The entry point was a no-op stub until this branch made it actually resize, at
which point it silently became device-only: it hands rgba_input/rgba_output
straight to NPP. A caller written against the old doc, which named them only
"Input [in_height, in_width, 4] as uint8", passes host buffers and gets an
illegal device access -- a sticky error that takes the CUDA context down for the
whole process rather than failing the call. Nothing in the signature or the doc
said so.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
(1 << eff_exp) * H * W * 4 was computed entirely in int. At 4096x4096 with
interpolation_exp 4 that is 2^36 bytes: the multiply overflows, the function
returns a NEGATIVE int, and a caller that feeds it straight to a size_t capacity
parameter converts it to a value near SIZE_MAX -- which sails through the
check_out_capacity() guard the same commit added, defeating the whole point of
the _sized entry points. The geometry is reachable: nothing clamps H/W, and
LIBREDIFFUSION_RIFE_MAX_EXP is well above 4.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
…lity

check_frame_sizes() rejected in_bytes != need / out_bytes != need / ehs_elements
!= need_ehs, so a caller who legitimately over-allocated -- a reusable scratch
buffer sized for the largest engine in a set, a std::vector grown by push_back --
got INVALID_DIMENSIONS for a call that would have been perfectly safe. The
entry points read exactly `need` bytes and write exactly `need` bytes; only a
SHORT buffer can hurt anyone. rife's check_out_capacity() already had it right;
this makes the two agree.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
librediffusion_img2img_turbo_ehs_elements() returned a compile-time constant
(77*1024) while the header told callers the sizes come from the engine, and
frameWidth()/frameHeight() were a hardcoded 512 for the same reason. Anyone
sizing a buffer from the documented contract on a non-512 or non-SD2.1 export
would be told the wrong number by the very function whose job is to be right.

Both are now discovered at construction from statically-shaped inputs -- the VAE
encoder's "image" [1,3,H,W] and the UNet's "ehs" [1,seq,dim] -- looked up through
an io_shape() helper so an engine missing the name does not emit a TensorRT error
line. The historical 512x512 / 77x1024 values remain the fallback for a dynamic
or unrecognised shape, so every existing bundle behaves identically. A VAE input
that is not a multiple of 8 is refused: the skip activations are H, H/2, H/4, H/8.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
jcelerier and others added 10 commits July 25, 2026 13:00
checked_malloc(), added on this branch to turn a silent null cudaMalloc into an
exception, gave the function three throw points with live allocations behind
them: a throw at "embeddings" leaks d_embeds1/d_embeds2/d_pooled_row, one at
"pooled_embeds" leaks those plus the embeddings buffer, one at "time_ids" leaks
embeddings + pooled_embeds. The caller sees a std::runtime_error and has nothing
to free -- SDXLPromptEmbeddings is three raw pointers and was never constructed.
Under the VRAM pressure that makes cudaMalloc fail in the first place, the node
retries per prompt change, so each attempt burns more of the memory it is short of.

Every allocation is now held by a scope guard and ownership is transferred to the
return value only past the last throw point. No behavioural change on success.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
…ection

shape_rejection() called eng->getTensorShape(tensor) directly on names that many
engines do not declare -- "images" on a bundle with no VAE encoder input by that
name, "latent", "sample". TensorRT logs an error line through the global logger
for every such call, so simply reinitialising a pipeline printed a burst of
scary-looking TensorRT errors for engines that were entirely fine, and the same
noise on every reinit. io_shape(), added in this PR for exactly this reason a
thousand lines further down, enumerates the IO tensors first and returns
nbDims = -1 for an absent name; shape_rejection already treats nbDims <= 0 as
"constrains nothing", so the behaviour is unchanged and only the noise goes away.
Moved to the top of the file so both users share one definition.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
…ument errors

All five validation throws added to prepare_scheduler on this branch were
std::runtime_error, so try_catch_wrapper collapsed them to -99 INTERNAL --
the same code a TensorRT enqueue failure or a corrupt engine produces. A caller
that passed mismatched span lengths, an empty schedule, a step count the pipeline
was not reinitialised for, a NaN coefficient or a zero alpha could not tell "I
called this wrong" from "the library broke", and had no reason to look at its own
arguments.

Introduces two exception types that mean exactly that (invalid_argument_error /
invalid_dimensions_error) and maps them at the C boundary, so length mismatches
report INVALID_DIMENSIONS and bad values report INVALID_ARGUMENT. Nothing else
throws them yet, so no other entry point changes its return code.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
librediffusion_pipeline_reinit_buffers() ran geometry_rejection() itself and then
called reinit_buffers(), which runs it again. That is two full sweeps of every
loaded engine's optimization profiles -- VAE encoder, VAE decoder, UNet, each with
a getProfileShape() pair per dynamic dimension -- on a path the node takes on every
resolution change, to reach exactly the same verdict.

The duplicate existed only because the C-API needed INVALID_DIMENSIONS and the
throw collapsed to INTERNAL. Now that reinit_buffers throws invalid_dimensions_error
the wrapper produces that code on its own, so the pre-check goes away.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
…NTERNAL

librediffusion_encode_image_half / _float / _decode_latent validated the handle
and the pointers and then went straight in, unlike every other inference entry
point. On a pipeline whose engines have not been brought up -- construction
succeeds long before init_engines() -- LibreDiffusionPipeline::encode_image()
calls vae_encoder_->encode() through a null unique_ptr. The caller gets -99
INTERNAL at best, where the sibling entry points all report -6 for exactly the
same "you called this too early" mistake.

The check is deliberately NOT check_inference_ready(): these entry points need
neither conditioning nor a schedule (the add-noise step is already guarded on
alpha_prod_t_sqrt_host_ being populated), and requiring prepare_embeds here would
break the standalone VAE-seam validation the harness does. encode_readiness() /
decode_readiness() assert only the engine each one actually drives.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
…(L-03)

librediffusion_img2img_turbo_frame_bytes() gives H*W*4, which is enough to size a
buffer but not enough to rescale an input frame or allocate an output texture:
512x512 and 256x1024 are the same number. The score node consequently hardcoded
512x512, which is precisely why it still overruns m_i2it_out on a non-512 engine
despite the _sized entry points existing.

librediffusion_img2img_turbo_frame_size(h, &w, &h) reports the same geometry the
buffer sizes are derived from, so a caller can be correct without guessing.
Optional symbol; older .so builds simply do not have it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
F-02 widened init_noise_ to max(noise_batch, batch_size, timestep_extent()) so the
consumers that read it at the latent extent stay in bounds. set_init_noise kept
copying init_noise_->size(), which is now that widened extent, out of a caller
buffer holding only the documented [use_denoising_batch ? denoising_steps :
batch_size, 4, lh, lw] — a device-side read past the end of memory the library does
not own.

Reproduces on every use_denoising_batch=false, steps>1 configuration: the golden
matrix's four hyper-sd15 txt2img batch-0 cells went PASS -> ERROR with C-API -4 out
of librediffusion_set_init_noise. The contract extent is now recorded at
init_buffers() time and the copy is clamped to it; the widened tail keeps the PCG
noise reseed() put there, and no configuration reads it.

Verified: test_init_noise (new) set_rc -4 -> 0 on the nobatch-2step / nobatch-4step
cases, batch-2step / batch-1step unchanged. The 4 matrix cells no longer error.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
…e.py does

3175947 also gated the encode-time noise, on the premise that the goldens were
unchanged. They were not: the matrix varies noise-0/noise-1 explicitly, and 73
img2img cells regressed — decode PSNR median 43.45 -> 19.92 dB, encode cosine down
to 0.065, SSIM negative on several.

src/streamdiffusion/pipeline.py calls add_noise from three places and the flag
guards two:
  :934 encode_image      unconditional
  :962 batched branch    gated, inside denoising_steps_num > 1
  :998 non-batched loop  gated, inside idx < len(sub_timesteps) - 1
so it means "re-noise the latent between denoising steps", and is inert at one step
by construction. The goldens confirm it: all 31 s1 noise-0/noise-1 pairs are
bit-identical, and at s2 the encode boundary matches while only the inter-step
tensors differ.

Both C++ inter-step seams already matched (formula, gate placement, and the
seed+idx+1 per-step noise); only encode_image diverged. It is unconditional again.
The alpha-scaled no-noise form the commit introduced was the reference's own else
branch, just applied at the wrong seam.

Verified: the 1-step img2img on-vs-off MAD is 0.00 again, and non-zero at 2 steps.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
config_set_dimensions bounds width/height at 16384, and its comment claimed that
also kept batch*4*lh*lw inside an int. It does not: nothing bounds batch_size,
denoising_steps or frame_buffer_size, and eight sites compute the latent element
count as int. At the maximum resolution batch_size 128 already crosses INT_MAX.

geometry_rejection could not catch it either — it computed unet_batch in int from
the same unbounded values, so the guard wrapped on exactly the input it exists to
reject — and it runs only from reinit_buffers, never on the first init_all.

init_buffers now evaluates the products in int64_t against INT_MAX and throws
invalid_dimensions_error naming the count; geometry_rejection computes in int64_t;
the comment states what the dimension bound actually covers.

Verified on origin/main: batch 128 at 16384x16384 CREATES the pipeline (rc=0) and
the process then dies. Fixed: refused at init with
"needs 2147483648 elements, which does not fit in int". test_intoverflow covers
batch 128 / 4096 at max resolution, 256 steps, and 2^29 batch at 512x512.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
Only the pipeline could be placed on a device, via config_set_device, and even that
held only on the thread that created it: cudaSetDevice is thread state and was
called once, in init_cuda. clip / rife / flux2 / flux2_stream / img2img_turbo took
no device at all and inherited whatever the calling thread happened to be on.

Every *_create now takes an explicit device, validates it against
cudaGetDeviceCount, and records it on the handle. Every entry point that touches
CUDA scopes the current device to the handle's for the call and restores it after
(DeviceGuard, device_guard.hpp), so a handle built on one thread is correct when
driven from another — the case a host's render/worker thread hits today.

The engine cache is keyed by (path, device): an ICudaEngine belongs to the device
it was deserialized on, so the same file on two devices must be two entries.
getCachedEngine reads the current device, which the guard has already set, so no
caller changed.

Signatures changed rather than adding _on_device variants (pre-1.0); the loader
derives its pointer types from the header so it needed no edit, and the six
harnesses pass 0.

Verified on a 2-GPU box (CUDA 0 = RTX 4090 sm_89, 1 = Quadro RTX 4000 sm_75 —
note CUDA's FASTEST_FIRST order is the REVERSE of nvidia-smi here): all 7 create
paths refuse an out-of-range ordinal without crashing, a valid ordinal reaches
that device (TRT then reports the arch mismatch, which is the engine's business),
and the guard restores the caller's device. lrd-hardening 38/38; golden cell
unchanged at cos 0.999991 / 56.91 dB.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzGZ1mL1wD1DQQ5fK6qRdr
@jcelerier
jcelerier force-pushed the fix/robustness-hardening branch from 41ed090 to f820e4a Compare July 26, 2026 03:07
@jcelerier
jcelerier merged commit b966e96 into main Jul 28, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant