MODEL-MM-GLM53-FLASH W9c-3b: the --device cuda SIGSEGV is a KV-page residency defect, not a mixed-residency one by localai-org-maint-bot · Pull Request #2483 · mudler/vllm.cpp

localai-org-maint-bot · 2026-09-01T14:03:22Z

MODEL-MM-GLM53-FLASH W9c-3b: the --device cuda SIGSEGV is a KV-page residency defect, not a mixed-residency one

--device cuda died with SIGSEGV on the published 101.24 GiB UD-Q2_K_XL
artifact, three legs of three, emitting no token. Spec O46 read the crashing
legs' log order — one device-arm announcement, then one fallback warning, then
death — as evidence that the fault is in the mixed device/host residency state
the per-layer fit guard creates. Both of those lines are static bool said
once-flags (glm5_next_moe.cpp, AnnounceDeviceArmOnce and
WarnDeviceFallbackOnce), so their order says only that at least one layer
staged and at least one LATER layer did not. It is not a location, and the
process died somewhere else.

It died in StoreCaches, host-storing into cudaMalloc memory.

GPUModelRunner::initialize_kv_cache resolves
kv_cache_backend_resident_ = !platforms::GetPlatform(dev.type).is_cpu() && (VT_DEVICE_KV_CACHE != "0")
(v1/worker/gpu/runner.cpp:1119-1122). On --device cuda it is true, and its
own comment says the predicate is deliberately "has a device", not "is CUDA".
CacheBuffer then allocates every paged cache and every recurrent state with
vt::Alloc(device_, ...) (runner.cpp:575-593). The host_data_ vector is
the CPU-queue arm and is not taken.
A cudaMalloc pointer is not host-dereferenceable on GB10 either.
src/vt/cuda/cuda_backend.cu:354-391 holds CUDA to the inherited
DeviceMemoryIsHostAddressable() == false with a static_assert, and names
The vt reference-tier fallback SEGFAULTS on device tensors instead of refusing by name, and a CUTLASS-less build reaches it silently #844 and Block-wise FP8 on a CUDA build without CUTLASS segfaults instead of refusing by name, and configure reports the feature ENABLED #1435 as the SIGSEGVs that came from reading the wide predicate
(UnifiedMemory()) instead.
glm5_next_kv.cpp read and wrote those pages with plain host loops —
static_cast<float*>(kv.data)[i] and t.Ptr<float>()[i]. LoadCaches and
StoreCaches were their only callers, and
ForwardGlm5NextForConditionalGeneration is the only caller of those two.

LoadCaches returns before touching a page on a fresh sequence
(if (b.cached_len <= 0) return;), so on step 1 the first host access to device
memory is inside StoreCaches, after the whole forward has returned. That is
after the announcement, after the fallback warning, on a step that emitted no
token, with no message — the recorded signature exactly.

The defect is older than the arm that exposed it. Nothing in W9c-3a's diff
touches a KV page; origin/main never reached this because
Glm5NextHostForward refused a non-CPU queue several thousand instructions
earlier. O46's three eliminated hypotheses are all inside MoeExpertsKeepQuant
and are all correctly eliminated — they were aimed at the wrong subsystem,
because the ordering that pointed them there was an artifact of two once-flags.

The change

src/vllm/model_executor/models/glm5_next_kv.cpp only. Every span this file
reads or writes now moves through vt::Backend::Copy instead of being
dereferenced. Every such span was already contiguous — a paged row is
head_size elements at PagedRowOffset(...) and a recurrent state is a whole
slot — so this is a substitution and not a redesign.

Backend::Copy is direction-agnostic on CUDA (cudaMemcpyAsync with
cudaMemcpyDefault, cuda_backend.cu:116-118), so ONE path is correct whether
the pages are device memory or host memory. That is deliberate: this file never
asks where the pages are, so it does not re-derive the runner's residency policy
and VT_DEVICE_KV_CACHE=0 needs no second branch here. On a CPU queue there is
no backend to ask and the direct memcpy is kept, so every --device cpu run
is byte-for-byte unchanged.

The gate, and the hole it fills

ResolveKvBinding, LoadCaches and StoreCaches had zero call sites
anywhere in tests/ before this PR. The engine binding W5b-2c landed was
exercised only incidentally, through ModelRegistry::Forward on a CPU queue,
where every page IS host memory and this defect cannot appear. That is how a
host-only page reader landed.

Two new cases drive all three functions over this suite's own three-group
Topology fixture, with the pages re-homed onto a ShadowBackend whose
allocations are not host-readable: the pointer Alloc hands back is a decoy
filled with a poison pattern and the real storage lives in a side block only
Copy can reach. Dereferencing it is therefore a wrong VALUE rather than a
fault a test binary cannot survive, and the assertion is deterministic on every
platform. The second case pins the other direction — a CPU queue must reach
Copy zero times and must still write the bytes in place at the permuted slot.

Measured, on x86_64 (this box), at `76a79d352`

suite	cases	assertions
`test_glm5_next_forward`	30	5,932
`test_glm5_next_moe`	18	12,731
`test_glm5_next_bridge`	20	32,562
`test_glm5_next_gguf_load`	16	8,731
`test_glm5_next_scaffold`	38	2,652
`test_glm5_next_layer`	10	1,656
`test_glm5_next_dsa`	10	1,934
`test_glm5_next_kda`	28	342
`test_glm5_next_attn`	14	160
`test_glm5_next_mhc`	5	98
`test_glm5_next_kpool_device`	4	4

test_glm5_next_forward was 28 cases before this PR; the two W9c-3b cases are
the difference. Both lines of every doctest summary are read, because a thrown
case does not add to the assertion tally (O48).

Sibling inertness, same box: test_mla_attention_block 21 / 2,282,067,
test_glm_moe_dsa_schedule 12 / 533, test_glm_moe_dsa_forward 7 / 5,258,
test_deepseek_v4_forward 6 / 34, test_deepseek_v4_paged_equiv 21 / 170,632,
test_dots3_note_attn 51 / 6,888, test_kimi_linear_paged 8 / 206,
test_kimi_linear_forward 15 / 875 — all cases passed, none thrown.
test_glm_moe_dsa_forward first read 7 cases / 127 assertions with one
REQUIRE failing, and that was the harness and not this change: the suite's
VT_MOE_EXPERT_STREAM=1 is a ctest ENVIRONMENT property
(tests/CMakeLists.txt:3720) and the binary had been run directly. With it set
the suite is 5,258 assertions and green — a 41x difference in the tally, which
is the same shape as a skip wearing a pass.

Mutations, each applied to product code and rebuilt:

#	mutation	result
M1	`PageIo::Write` always takes the direct `memcpy` arm	KILLED — 1 case / 5,637 assertions
M2	`PageIo::Read` always takes the direct `memcpy` arm	KILLED — 1 case / 2,818 assertions
M3	`PageIo` drops its CPU early return, so a CPU queue bounces too	SURVIVED
M4	delete the production `StoreCaches` call in `glm5_next_registry.cpp`	KILLED — 2 cases / 3 assertions (the REACHABILITY mutation)

M3's survival is reported rather than papered over, and the case now says what
it can and cannot see: the CPU backend's Copy IS std::memcpy, so the bytes
are identical either way and the counter watches the shadow backend a CPU queue
never reaches. The early return is not load-bearing for correctness; it keeps a
build with no CPU backend from Failing on the host path.

M4 is the reachability answer: ModelRegistry::Forward →
ForwardGlm5NextForConditionalGeneration → StoreCaches, and deleting that one
line reds the two W5b-2c cases that read the pages back on a second step. The
two W9c-3b cases do NOT red under M4 — they call the three functions directly,
they localise the defect, and they are deliberately not the reachability proof.

The tree was restored byte-for-byte after every mutation (sha256 compared, and
the suite re-run green afterwards).

Hardware, run 1 on `dgx:gpu0` (GB10, `sm_121a`) — the diagnosis is CONFIRMED

leg	binary	device	env	rc	wall	stdout
B	base `3cd467643`	cuda	default	139	—	0 bytes
A	base `3cd467643`	cuda	`VT_DEVICE_KV_CACHE=0`	0	1756 s	`Paris Paris`

Leg B's backtrace names the predicted frame and the whole chain:

Thread 4 "vllm-cli" received signal SIGSEGV, Segmentation fault.
#0  vllm::glm5_next::StoreCaches(...)
#1  vllm::(anonymous namespace)::ForwardGlm5NextForConditionalGeneration(...)
#2  vllm::ModelRegistry::Forward(...)
#3  vllm::v1::GPUModelRunner::execute_model(...)

Leg A moved the KV pages into host memory and changed nothing else — same
binary, same box, minutes later — and survived. A predicted frame plus a
one-variable falsification that fails to falsify are two independent
confirmations.

Legs C and D did not run, and the job's own guard is why — with a correction

The guard fired and refused to report them. It was right to refuse and wrong
in what it hashed: it compared sha256sum vllm-cli across the two halves.

artifact	base	fixed
`examples/vllm-cli`	`bbc55a61…`	`bbc55a61…`
`libvllm.so.0.0.3`	`bfd0a6e7…`	`66412203…`

vllm-cli is a thin ABI client this change cannot touch, so its hash is
identical by construction; the model code is in libvllm.so, and those
hashes differ. Two genuinely different binaries were built. Separate build
directories would not have fixed this — vllm-cli hashes the same across them
too — so the repair is the predicate: digest the executable and every .so
beside it, the breadth the same script's identity scan already used. Run 2
does that, uses separate build directories anyway so incremental staleness is
not even a question, and adds a source-level sentinel (a string only the patched
file contains) checked in the linked set.

The second token, and a discriminator that cannot run

Leg A emitted Paris Paris. Token 1 agrees with the CPU arm, token 2 does not:
prefill is clean and the decode step is not. (PARIS_A=NO is a string match
against Paris. and is an artifact of that flag, not the finding.)

Unsetting VT_GLM5_NEXT_DEVICE_EXPERTS cannot discriminate anything, and
this is verified in source rather than assumed: with the flag unset,
Glm5NextHostForward refuses a non-CPU queue by name before any token is
produced. The leg would emit nothing, which at the file level is
indistinguishable from the crash it would be mistaken for.

The candidate with a mechanism is one this PR already flagged as unestablished.
GPUModelRunner::async_device_mirror() is default ON for an integrated CUDA
GPU (runner.cpp:4437-4459; GB10 satisfies is_integrated_gpu()), and on that
path the combine patches the device input ids and leaves the host token_ids
"deliberately stale for decode rows" — the runner's own words
(runner.cpp:2374-2414, assigned :2748). This model references
device_token_ids zero times in either forward file. On a decode step that
is a stale id, so the model never sees the token it just emitted — which is what
re-emitting Paris looks like, and why prefill is unaffected.

Run 2's legs: D (fixed, cpu — the operand, taken first), C (fixed, cuda,
default KV — the merge criterion), E (VT_ASYNC_DEVICE_MIRROR=0, the
documented rollback — the decode discriminator), F (--max-tokens 1,
prefill vs decode by construction), G (base + VT_DEVICE_KV_CACHE=0, run
1's leg A repeated within run 2). Queued as a776deb7 on dgx:gpu0.

A bimodal reading is required: the unit gate is NMSE 3.833e-15 and the
prefill argmax margin is 1.279, so a perturbation far too small to move token 1
can still flip a near-tie at token 2. "Token 1 matched" does not clear the expert
arm on its own.

What is NOT claimed yet

The device arm stays OPT-IN and defaults OFF. The FIX itself is still
unmeasured on hardware — leg C does not exist yet, and the merge criterion is
RC_C=0 with output byte-matching the CPU arm. What run 1 established is the
CAUSE, not the cure. No speed number is admissible from any leg (O47). The job is staged at
/workspace/glm53-kvres/run.sh and queued on dgx:gpu0; it takes a gdb
backtrace of the reproducing leg as well, because a host SIGSEGV is invisible to
compute-sanitizer.

Every number above came from this x86_64 development box. Nothing in this PR
was measured on a GPU.

Closes #2480.

FOLLOWING_AGENTS_PROTOCOL

Refs: #2464, #2410

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]

…ifact of two once-flags O46 reads the crashing legs' log order -- one device-arm announcement, then one fallback warning, then SIGSEGV -- as evidence that the fault is in the mixed residency state the per-layer fit guard creates. Both of those lines are `static bool said` once-flags, so their order says only that at least one layer staged and at least one later layer did not. It is not a location, and the process died somewhere else. It died in `StoreCaches`. On `--device cuda` the runner sets `kv_cache_backend_resident_` from `!platform.is_cpu()` and allocates every paged cache and every recurrent state with `vt::Alloc` -- `cudaMalloc` -- while `glm5_next_kv.cpp` reads and writes those pages with plain host loops. A `cudaMalloc` pointer is not host-dereferenceable on GB10 either, which `cuda_backend.cu` holds with a `static_assert` naming #844 and #1435 as the same fault twice before. `LoadCaches` returns before touching a page on a fresh sequence, so the first host access to device memory on step 1 is after the whole forward has returned: after the announcement, after the fallback warning, on a step that emitted no token. W9c-3a did not introduce it. `origin/main` refused a non-CPU queue several thousand instructions earlier, and removing that refusal made a pre-existing hole reachable. O46's three eliminated hypotheses are all inside `MoeExpertsKeepQuant` and are all correctly eliminated; the ordering that pointed them there was the artifact. This commit is the wave block and O49 only. The change it scopes is the next commit, and the on-box legs that discriminate the diagnosis are named here before either. FOLLOWING_AGENTS_PROTOCOL Refs: #2480, #2464, #2410 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

…evice queue they are not host memory `--device cuda` died with SIGSEGV on the real 101.24 GiB artifact, three legs of three, emitting no token. The fault is a host store into `cudaMalloc` memory in `StoreCaches`, and it is older than the device-expert arm that exposed it. On any non-CPU queue the runner sets `kv_cache_backend_resident_` from `!platform.is_cpu()` and allocates every paged cache and every recurrent state through `vt::Alloc` (`v1/worker/gpu/runner.cpp:575-593`, `:1119-1122`). On CUDA that is `cudaMalloc`, and `cuda_backend.cu:354-391` holds CUDA to `DeviceMemoryIsHostAddressable() == false` with a `static_assert` naming #844 and #1435 as the two SIGSEGVs that came from believing the wide predicate instead. This file read and wrote those pages with plain host loops. Every span it addresses is contiguous already -- one paged row is `head_size` elements at `PagedRowOffset`, and a recurrent state is a whole slot -- so the change is a substitution rather than a redesign: build or read the span in a host staging buffer and move it with `vt::Backend::Copy`. `Copy` is direction-agnostic on CUDA (`cudaMemcpyDefault`), so ONE path is correct whether the pages are device memory or host memory, and this file therefore never has to re-derive the runner's residency policy or read `VT_DEVICE_KV_CACHE`. A CPU queue has no backend to ask and keeps the direct `memcpy`, so every `--device cpu` run is byte-for-byte unchanged. `ResolveKvBinding`, `LoadCaches` and `StoreCaches` had ZERO call sites anywhere in `tests/` before this commit, which is how a host-only page reader landed in the engine binding at all. The new cases drive all three over this suite's own three-group topology with the pages re-homed onto a backend whose allocations are not host-readable: the pointer `Alloc` returns is a decoy full of poison and the real storage sits in a side block only `Copy` can reach, so dereferencing it is a wrong VALUE instead of a fault a test binary cannot survive. The second case pins the other direction -- a CPU queue must reach `Copy` zero times and must still write the bytes in place. The opt-in default stays OFF until the four `dgx:gpu0` legs the spec names have run, because a fix that has not been driven on the artifact that produced the crash is a hypothesis. FOLLOWING_AGENTS_PROTOCOL Refs: #2480, #2464 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

…en, not only where it is replaced O49 says the mixed-residency reading of the three SIGSEGV legs is wrong. O46 and the `## Now` block still asserted it in their own words, and a record that contradicts itself is read by whoever opens it first rather than by whoever opens it last. O46 keeps its measured table and its three eliminated hypotheses, which are sound; the one sentence that turned a log ORDER into a fault LOCATION is retracted in place with the reason, and points at O49. `## Now` loses the same clause. FOLLOWING_AGENTS_PROTOCOL Refs: #2480, #2464 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

…use a mutation proved it sees less Dropping `PageIo`'s CPU early return -- so that a CPU queue resolves the CPU backend and bounces through it too -- SURVIVES the whole suite. It has to. The CPU backend's `Copy` IS `std::memcpy`, so the bytes are identical either way, and the case's counter watches the SHADOW backend, which a CPU queue never reaches under either version. So the early return is not load-bearing for correctness, and the case no longer claims it is. It is there so a build with no CPU backend registered does not `Fail` on the host path, and so the `--device cpu` instruction stream stays the one that was already measured. The case now asserts what it can see -- the bytes land in the topology's own host vectors at the permuted slot -- and names the surviving mutation in its own comment. The spec gains the mutation table with what each one killed, M3's survival included, and the note that the staging is per ROW rather than per block: about 180,000 round trips to hydrate a full 8192-token prefix where block coalescing would be about 5,600. Not done here, because this forward is a host reference at roughly 85 s per step and no speed number is admissible from this row, but written down rather than left to be rediscovered. FOLLOWING_AGENTS_PROTOCOL Refs: #2480, #2464 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

…chanism, and it is false `docs/ENVIRONMENT.md`'s row for `VT_GLM5_NEXT_DEVICE_EXPERTS` and `docs/FEATURES.md`'s GLM-5.3-Flash row each told a user the `--device cuda` crash "appears only in the MIXED residency state the per-layer fit guard creates". A record correction that leaves the same claim in the product's own documentation has corrected nothing a user reads. Both now say what was measured -- three of three legs die, the knob stays off -- and then name the actual fault site, `StoreCaches` host-storing into the runner's `cudaMalloc` KV pages after the forward has already returned, with the issue that owns the fix. The default and the warning are unchanged: this is not a claim that the knob is safe. FOLLOWING_AGENTS_PROTOCOL Refs: #2480, #2464 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

…be the fault, not only that it is not O49 named a different fault site. That leaves the reader free to think both stories are live and this one merely arrived first, so the argument is completed where it is made. The two arms of `MoeExpertsKeepQuant` share no mutable state across layers except `OwnedTensor::d_dev`, which is per tensor. `DeviceBanksFit` returns true whenever nothing needs uploading, so a resident layer always takes the device arm and never the host one, and a layer that falls back is one whose banks were never staged -- its host views aim at bytes staging did not touch, because `AdoptDeviceBytesAsHost` returns early on CUDA. That last clause is O46's own hypothesis 2, which it killed correctly and then did not carry forward into the mechanism it kept. There is no interleaving of staged and unstaged layers that lets one arm read the other's memory. FOLLOWING_AGENTS_PROTOCOL Refs: #2480, #2464 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

…measures The async runner can hand a forward device-resident input ids and leave `token_ids` stale for decode rows. This model reads `token_ids` and ignores the device field. Whether that path engages here is NOT established -- it needs `async_input_combine_`, and this row has never observed a decode step on a device queue -- so it is not filed as a defect and no fix is written for it. It is named beside the evidence table because the legs already answer it: two tokens is exactly one decode step, so a leg C whose stdout matches leg D byte-for-byte has measured it, and a leg C that emits ` Paris` and then the wrong second token has found it. Writing that down is what stops the next reader from either assuming it is fine or spending a lease to ask separately. FOLLOWING_AGENTS_PROTOCOL Refs: #2480, #2464 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

Six commits, none of them touching `glm5_next_kv.cpp`, the glm5_next tests, or the runner's KV residency decision. The largest is DeepSeek-V4's EXL3 routed experts running on CUDA on a default configuration, which is the same shape of change one row over and corroborates this branch's reading rather than conflicting with it: that model hands its paged cache to DEVICE kernels and never dereferences it on the host. Taken so the pull request is measured against what actually ships. The suite counts in the body were taken at `76a79d352`; the glm5_next suites are re-run on this merge commit below. FOLLOWING_AGENTS_PROTOCOL Refs: #2480, #2464 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

…he job's own guard hid the fix Run 1 on `dgx:gpu0` settles the cause two independent ways. Leg B died with SIGSEGV and its backtrace names the predicted frame and the whole chain -- `StoreCaches` under `ForwardGlm5NextForConditionalGeneration` under `ModelRegistry::Forward` under `execute_model`. Leg A moved the KV pages into host memory with `VT_DEVICE_KV_CACHE=0`, changed nothing else, and survived at rc=0 in 1756 s on the same binary minutes later. A predicted frame plus a one-variable falsification that fails to falsify retires O46's mechanism rather than merely doubting it. Legs C and D never ran, and the job refused them itself: it compared `sha256sum vllm-cli` across the two halves, they matched, and it declared them the same binary. They were not. The `.so` hashes the same script recorded differ -- `bfd0a6e7` base against `66412203` fixed -- and `vllm-cli` is a thin ABI client this change cannot alter, so its hash is identical by construction. Separate build directories would not have fixed that either. The repair is the predicate: digest the executable and every `.so` beside it, which is the breadth the same script's identity scan already used two blocks earlier. Run 2 does both and adds a source-level sentinel checked in the linked set. O52 opens the second-token question rather than closing it. Leg A's first token agrees with the CPU arm and its second does not, so prefill is clean and decode is not. The candidate with a mechanism is the runner's async device mirror, default ON for an integrated CUDA GPU, which leaves the host `token_ids` stale for decode rows while this model references `device_token_ids` zero times. That is a candidate, and the entry says so. It also records why the obvious discriminator cannot be run: with `VT_GLM5_NEXT_DEVICE_EXPERTS` unset the forward refuses a non-CPU queue by name and emits no token at all. FOLLOWING_AGENTS_PROTOCOL Refs: #2480, #2464, #2410 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

… 1 settled and what run 2 must answer Legs A and B are taken and are recorded as taken, so the table no longer asks for them. What replaces them is the set run 2 needs: D first, because every byte comparison needs an operand from the same binary on the same box and the ` Paris.` in the records came from a different build; C, which is the merge criterion; and E, F and G, which separate the second token. The stop conditions gain the one this row was missing. A leg C that emits a token but does not match D is TWO results -- the crash is fixed and the second token is wrong -- and the first must not be reported without the second beside it. FOLLOWING_AGENTS_PROTOCOL Refs: #2480, #2464 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

…ailed a criterion I mis-specified Run 2 completed on dgx:gpu0 at 2026-09-01T21:56:05Z and its results were never written back here; this spec still described those legs in the future tense. The crash is fixed: run 1 leg B died rc=139 with StoreCaches at frame #0, and leg C is the same workload on the same box at rc=0, on a pair whose linked-set digests genuinely differ. The stated criterion "C byte-identical to D" failed, and the fault is in the criterion rather than the fix. C and D differ by two variables, where the KV pages live and whether the async device mirror is engaged, and a StoreCaches fix can only be held to the first. Against the single-variable operand, leg G on the base binary and host pages, leg C is byte-identical. The clause stays red and nothing was deleted to make it green. The second token is a separate defect that predates this fix. Leg G is a base build and diverges identically; leg E flips VT_ASYNC_DEVICE_MIRROR alone and the divergence disappears into byte-identity with the CPU operand. That is the pre-registered read-key, and it promotes #2544 from a grep-based candidate to a measured conviction for this model. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

mudler added 10 commits September 1, 2026 13:57

localai-org-maint-bot mentioned this pull request Sep 2, 2026

test_mla_attention_block's 3 'failures' on dgx are THROWN CUDA out-of-memory, not a defect: HasCuda() gates on backend existence, not capacity #2571

Open

mudler merged commit b898d6f into main Sep 2, 2026
3 of 17 checks passed

Navigation Menu

MODEL-MM-GLM53-FLASH W9c-3b: the --device cuda SIGSEGV is a KV-page residency defect, not a mixed-residency one - #2483

MODEL-MM-GLM53-FLASH W9c-3b: the --device cuda SIGSEGV is a KV-page residency defect, not a mixed-residency one#2483
mudler merged 11 commits into
mainfrom
row/MODEL-MM-GLM53-FLASH-DEVSEGV

localai-org-maint-bot commented Sep 1, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

localai-org-maint-bot commented Sep 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The change

The gate, and the hole it fills

Measured, on x86_64 (this box), at 76a79d352

Hardware, run 1 on dgx:gpu0 (GB10, sm_121a) — the diagnosis is CONFIRMED

Legs C and D did not run, and the job's own guard is why — with a correction

The second token, and a discriminator that cannot run

What is NOT claimed yet

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

localai-org-maint-bot commented Sep 1, 2026 •

edited

Loading

Measured, on x86_64 (this box), at `76a79d352`

Hardware, run 1 on `dgx:gpu0` (GB10, `sm_121a`) — the diagnosis is CONFIRMED