GPU governance, resource observability, multimodal genome architecture - #281
Merged
Conversation
…m hot-reload Three bugs preventing trained adapters from influencing inference: - getEffectiveModel() now receives active adapter domain for Rust 4-tier model selection - scoreInteraction() assigns quality ratings to training data (enables 20-example micro-tuning) - Post-training callback uses AdapterStore filesystem discovery instead of empty DB layers array
…iminate build race Every GPU consumer is now visible to the pressure system: - GpuModelTracker interface compresses allocation boilerplate to one call per consumer - Orpheus TTS (LLM+SNAC), Pocket TTS, Piper TTS, Kokoro TTS all tracked - Embedding models tracked with size estimates for 7 known model variants - LoRA rebuild transient spikes tracked (auto-release on method exit) Safety fixes: - CandleAdapter returns error at critical pressure instead of loading anyway - LearningScheduler defers training at >60% GPU pressure (lowest priority workload) - TOCTOU race in allocate() fixed: optimistic-allocate-then-rollback pattern - Memory ordering upgraded from Relaxed to AcqRel for safety-critical atomics - GpuStats exports pressure thresholds (60/80/95%) so TS stops hardcoding Dead code removed: - gpu_allocator.rs deleted (478 lines, superseded by memory_manager.rs, zero callers) Build pipeline fix: - parallel-start.sh restructured: Rust build completes before TS prebuild starts - Eliminates cargo target directory contention that caused 300s binding gen timeout - Voice models still download in parallel with cargo build (no contention) - TS build + VRM conversion run in parallel after Rust completes
…ations GpuPriority enum (Realtime/Interactive/Background/Batch) with per-level pressure gates. Realtime (render loop, audio) only stops at 95% OOM. Batch (training) yields the bus at 50%. All 14 GPU callers updated with correct priority assignments. Per-priority allocation counters in GpuStats. 43 tests pass including 12 new priority-specific tests.
…mmand layers IPC mixin was silently dropping new GpuStats fields during snake_case→camelCase conversion. Updated all three layers: bindings/modules/gpu.ts, GpuStatsTypes.ts, GpuStatsServerCommand.ts.
…tem resources IPC
Layer 1 — GPU Eviction Registry (Rust):
- EvictionRegistry tracks all GPU consumers with priority, bytes, last-used timestamps
- Eviction scoring: age_seconds / (priority_weight * 10) for priority-aware ordering
- IPC commands: gpu/eviction-registry, gpu/eviction-candidates
- TS mixin: gpuEvictionRegistry(), gpuEvictionCandidates()
- Registered at all allocation sites: Candle, TTS engines, Bevy, embeddings, genome paging
Layer 2 — Pressure Watchers (TypeScript):
- GpuPressureWatcher: adaptive polling (10s/3s/1s/500ms), Events on threshold crossings
- ResourcePressureWatcher: CPU + memory monitoring, same adaptive pattern
- Events: gpu:pressure:{level}, cpu:pressure:{level}, memory:pressure:{level}
- Migrated LearningScheduler + GenomeTrainServerCommand from inline polling to watchers
System Resources Module (Rust + TypeScript, full pipeline):
- SystemResourceMonitor: sysinfo 0.33, cached readings, explicit refresh
- IPC commands: system/cpu, system/memory, system/resources
- Process CPU baseline fix: double-refresh on first query for accurate deltas
- TS IPC mixin: systemCpu(), systemMemory(), systemResources()
- ./jtag system/resources command: generated scaffold, wired to IPC mixin
- 22 Rust tests, all passing
Verified live: CPU 87% (M1 Pro), memory 64%, top processes with real CPU %,
GPU eviction candidates sorted by priority, pressure events firing in logs.
…n control Maps the full stack from priority-gated allocation (Rust, done) through pressure watchers (TS, done) to the planned GpuGovernor mechanical strategy and LoRA-trained Qwen sentinel for AI-driven resource decisions. Covers telemetry frame design, decision output schema, training data self-generation from pressure events, and sentinel pipeline definition.
…osystem design Updates architecture doc with specific Qwen 3.5 Small Model Series (0.8B/2B/4B/9B) for tiered workloads from always-loaded governance sentinels to persona backbones. Adds multimodal LoRA concept (vision/audio/document adapters as genome layers), self-improving ecosystem vision, and Reticulum grid resource coordination.
The genome isn't just text personality — in a fully immersive environment with avatars, spatial audio, and real-time video, a persona's identity spans every modality. Adds Section 7 covering voice LoRA (Orpheus), vision LoRA (Qwen3.5-4B multimodal), generation LoRA (SDXL style), and governance LoRA (Qwen3.5-0.8B resource management). Each modality uses the same GenomeLayerEntity, AdapterStore, genome paging, and resource governance infrastructure. The interface is proven — we're adding implementations per modality.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a multi-layer resource governance/observability stack: priority-gated GPU allocations with an eviction registry + pressure watchers, plus a new system resources (CPU/memory/process) IPC/command surface, and expands the genome system with quantization metadata, conversion pipelines, capability embeddings, and fitness tracking.
Changes:
- Add GPU governance primitives (priority scheduling gates, eviction registry + IPC, shared tracking helper, pressure watcher integration).
- Add system resource monitoring end-to-end (Rust sysinfo monitor + IPC module, TS bindings,
system/resourcescommand, and a TS pressure watcher). - Extend genome learning lifecycle (QLoRA quantization metadata, adapter conversion pipeline/command, capability embeddings + registry search, and fitness persistence).
Reviewed changes
Copilot reviewed 92 out of 96 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| src/workers/shared/mod.rs | Removes old worker-level GPU allocator references; points to continuum-core GPU manager. |
| src/workers/continuum-core/src/system_resources/mod.rs | Adds system_resources module exports. |
| src/workers/continuum-core/src/runtime/runtime.rs | Registers system module name in expected module list. |
| src/workers/continuum-core/src/persona/genome_paging.rs | Adds GPU priority allocation + eviction registry register/unregister for adapters. |
| src/workers/continuum-core/src/modules/system_resources.rs | New IPC module for CPU/memory/resources snapshot (+ tests). |
| src/workers/continuum-core/src/modules/mod.rs | Exposes system_resources module. |
| src/workers/continuum-core/src/modules/gpu.rs | Adds eviction registry/candidates IPC commands (+ tests). |
| src/workers/continuum-core/src/modules/embedding.rs | Adds GPU allocation tracking + eviction registry entry for embedding models. |
| src/workers/continuum-core/src/live/video/bevy_renderer.rs | Adds GPU priority allocation + eviction registry registration for render targets/models. |
| src/workers/continuum-core/src/live/audio/tts/pocket.rs | Uses GpuModelTracker for Pocket-TTS allocation tracking + touch on use. |
| src/workers/continuum-core/src/live/audio/tts/piper.rs | Uses GpuModelTracker for Piper allocation tracking + touch on use. |
| src/workers/continuum-core/src/live/audio/tts/orpheus.rs | Uses GpuModelTracker for Orpheus model allocations + touch on use. |
| src/workers/continuum-core/src/live/audio/tts/kokoro.rs | Replaces single OnceLock guard with GpuModelTracker + touch on use. |
| src/workers/continuum-core/src/lib.rs | Exposes system_resources module from crate root. |
| src/workers/continuum-core/src/ipc/mod.rs | Wires GPU manager into embedding + registers SystemResourceModule. |
| src/workers/continuum-core/src/inference/vendored/quantized_llama.rs | Adds GGUF mixed-precision LoRA merge + parsing + tests. |
| src/workers/continuum-core/src/inference/candle_adapter.rs | Adds GPU priority allocations, eviction registry bookkeeping, and pressure-gated failures. |
| src/workers/continuum-core/src/inference/backends/mod.rs | Extends backend LoRA rebuild API to accept optional GPU manager. |
| src/workers/continuum-core/src/inference/backends/llama_safetensors.rs | Adds transient “spike” allocation tracking during rebuild_with_lora. |
| src/workers/continuum-core/src/inference/backends/llama_gguf.rs | Implements GGUF LoRA merge rebuild_with_lora + transient spike tracking. |
| src/workers/continuum-core/src/gpu/tracker.rs | New reusable GPU allocation lifecycle helper (GpuModelTracker) + tests. |
| src/workers/continuum-core/src/gpu/mod.rs | Re-exports eviction registry + priority/thresholds + tracker types. |
| src/workers/continuum-core/src/gpu/eviction_registry.rs | New eviction registry snapshot/candidate scoring + ts-rs exports + tests. |
| src/workers/continuum-core/bindings/modules/system_resources.ts | TS IPC mixin for system resources (type mapping + request helpers). |
| src/workers/continuum-core/bindings/modules/gpu.ts | Extends GPU TS mixin with thresholds, allocations-by-priority, eviction IPC. |
| src/workers/continuum-core/bindings/RustCoreIPC.ts | Composes SystemResourceMixin into client. |
| src/workers/continuum-core/Cargo.toml | Adds sysinfo dependency. |
| src/system/user/server/modules/being/LimbicSystem.ts | Hot-load adapters post-training, add adoptAdapter, recompute composite embedding. |
| src/system/user/server/modules/PersonaResponseGenerator.ts | Adds fitness tracking for successful inference calls. |
| src/system/user/server/modules/PersonaGenome.ts | Stores layerId on adapter; updates GPU-management comment. |
| src/system/user/server/modules/LoRAAdapter.ts | Adds layerId back-reference getter for fitness tracking. |
| src/system/user/server/PersonaUser.ts | Adds corpus reload retry after Hippocampus schema init. |
| src/system/sentinel/pipelines/GenomeConvertPipeline.ts | New sentinel pipeline to run genome/convert and register/activate output. |
| src/system/resources/server/ResourcePressureWatcher.ts | New CPU/memory pressure watcher (adaptive polling + Events). |
| src/system/rag/sources/ToolDefinitionsSource.ts | Adds “use tools” behavioral nudge in tool prompt sections. |
| src/system/rag/sources/SemanticMemorySource.ts | Shortens negative-cache TTL for “No memory corpus”. |
| src/system/rag/sources/PersonaIdentitySource.ts | Adds “use tools” nudge to identity system prompt sections. |
| src/system/rag/builders/ChatRAGBuilder.ts | Marks legacy system-prompt path deprecated + adds “use tools” wording. |
| src/system/gpu/server/GpuPressureWatcher.ts | New GPU pressure watcher (adaptive polling + Events). |
| src/system/genome/shared/AdapterPackageTypes.ts | Adds QuantizationInfo and manifest field. |
| src/system/genome/server/TrainingCompletionHandler.ts | Reads quantization info; generates capability embeddings; persists to DB. |
| src/system/genome/server/LearningScheduler.ts | Adds capability-based adoption check + GPU pressure gating for training. |
| src/system/genome/server/GenomeRegistry.ts | New capability embedding search over GenomeLayerEntity collection. |
| src/system/genome/server/FitnessTracker.ts | New debounced fitness persistence for genome layers. |
| src/system/genome/server/AdapterStore.ts | Adds quantization field to manifest typing. |
| src/system/genome/server/AdapterPackage.ts | Persists quantization; adds capability embedding generation helper. |
| src/system/genome/fine-tuning/server/adapters/scripts/peft-train.py | Writes quantization_info.json; adds 4-bit→8-bit fallback and metadata reporting. |
| src/system/genome/fine-tuning/server/BaseServerLoRATrainer.ts | Reads quantization_info.json and includes it in manifest + logging. |
| src/system/genome/entities/GenomeLayerEntity.ts | Embedding becomes optional/variable-dim with embeddingDimension + quantization field. |
| src/system/genome/entities/GenomeEntity.ts | Composite embedding becomes optional/variable-dim with embeddingDimension. |
| src/system/core/system/server/JTAGSystemServer.ts | Starts GPU + system resource pressure watchers at server start. |
| src/shared/version.ts | Version bump. |
| src/shared/generated/index.ts | Exports generated system module types. |
| src/shared/generated/gpu/index.ts | Exports new generated GPU types. |
| src/shared/generated/gpu/GpuStats.ts | Generated type includes thresholds + allocations_by_priority. |
| src/shared/generated-command-constants.ts | Adds constants for genome/convert and system/resources commands. |
| src/server/generated.ts | Generated server registry updated for new commands. |
| src/scripts/parallel-start.sh | Adjusts build pipeline ordering to avoid cargo contention. |
| src/package.json | Version bump. |
| src/package-lock.json | Version bump. |
| src/generator/specs/system-resources.json | Adds generator spec for system/resources. |
| src/generator/specs/genome-convert.json | Adds generator spec for genome/convert. |
| src/generator/generate-rust-bindings.ts | Documents binding gen sequencing expectations. |
| src/generated-command-schemas.json | Generated schema updates for new commands. |
| src/docs/genome/QLORA-QUANTIZATION.md | Adds QLoRA/quantization architecture + conversion + inference notes. |
| src/docs/genome/DYNAMIC-GENOME-ARCHITECTURE.md | Marks as superseded by GENOME-ARCHITECTURE.md. |
| src/docs/CONTINUOUS-LEARNING-RUNTIME.md | Marks as superseded by GENOME-ARCHITECTURE.md. |
| src/docs/COMPOSABLE-EXPERTISE.md | Marks as superseded by GENOME-ARCHITECTURE.md. |
| src/docs/COLLABORATIVE-LEARNING-VISION.md | Marks as superseded by GENOME-ARCHITECTURE.md. |
| src/commands/system/resources/test/unit/SystemResourcesCommand.test.ts | Adds generated placeholder unit test scaffold for system/resources. |
| src/commands/system/resources/test/integration/SystemResourcesIntegration.test.ts | Adds generated placeholder integration test scaffold for system/resources. |
| src/commands/system/resources/shared/SystemResourcesTypes.ts | Adds typed command params/result helpers for system/resources. |
| src/commands/system/resources/server/SystemResourcesServerCommand.ts | Implements system/resources server command via Rust IPC. |
| src/commands/system/resources/package.json | Adds command package metadata for system/resources. |
| src/commands/system/resources/browser/SystemResourcesBrowserCommand.ts | Adds browser delegating command for system/resources. |
| src/commands/system/resources/README.md | Adds command README for system/resources. |
| src/commands/system/resources/.npmignore | Adds publish ignore file for system/resources package. |
| src/commands/gpu/stats/shared/GpuStatsTypes.ts | Extends GPU stats command types for thresholds + allocations-by-priority. |
| src/commands/gpu/stats/server/GpuStatsServerCommand.ts | Returns new threshold/allocation fields from Rust GPU stats. |
| src/commands/genome/train/server/GenomeTrainServerCommand.ts | Uses cached GPU pressure watcher; generates capability embedding after training. |
| src/commands/genome/convert/test/unit/GenomeConvertCommand.test.ts | Adds genome/convert unit tests for validation/defaulting logic. |
| src/commands/genome/convert/test/integration/GenomeConvertIntegration.test.ts | Adds generated placeholder integration test scaffold for genome/convert. |
| src/commands/genome/convert/shared/GenomeConvertTypes.ts | Adds typed command params/result helpers for genome/convert. |
| src/commands/genome/convert/server/GenomeConvertServerCommand.ts | Implements genome/convert via sentinel subprocess + optional validation. |
| src/commands/genome/convert/package.json | Adds command package metadata for genome/convert. |
| src/commands/genome/convert/browser/GenomeConvertBrowserCommand.ts | Adds browser delegating command for genome/convert. |
| src/commands/genome/convert/README.md | Adds command README for genome/convert. |
| src/commands/genome/convert/.npmignore | Adds publish ignore file for genome/convert package. |
| src/browser/generated.ts | Generated browser registry updated for new commands. |
Files not reviewed (1)
- src/package-lock.json: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+45
to
+51
| ): SystemResourcesParams => createPayload(context, sessionId, { | ||
| userId: SYSTEM_SCOPES.SYSTEM, | ||
| includeProcesses: data.includeProcesses ?? false, | ||
| topN: data.topN ?? 0, | ||
| ...data | ||
| }); | ||
|
|
There was a problem hiding this comment.
In createSystemResourcesParams(), defaulting topN to 0 means callers who omit topN (the common case) will send topN=0 to the server, which then forwards it to Rust and yields an empty process list even though the API/docs imply a default of 10. Consider leaving topN undefined when not provided (so Rust can apply its default), or defaulting to 10 when includeProcesses=true, and avoid forcing 0 here.
Suggested change
| ): SystemResourcesParams => createPayload(context, sessionId, { | |
| userId: SYSTEM_SCOPES.SYSTEM, | |
| includeProcesses: data.includeProcesses ?? false, | |
| topN: data.topN ?? 0, | |
| ...data | |
| }); | |
| ): SystemResourcesParams => { | |
| const { includeProcesses: includeProcessesRaw, topN: topNRaw, ...rest } = data; | |
| const includeProcesses = includeProcessesRaw ?? false; | |
| const effectiveTopN = topNRaw ?? (includeProcesses ? 10 : undefined); | |
| return createPayload(context, sessionId, { | |
| userId: SYSTEM_SCOPES.SYSTEM, | |
| includeProcesses, | |
| ...rest, | |
| ...(effectiveTopN !== undefined ? { topN: effectiveTopN } : {}), | |
| }); | |
| }; |
Comment on lines
+56
to
+58
| bits: data.bits ?? 0, | ||
| outputPath: data.outputPath ?? '', | ||
| validate: data.validate ?? false, |
There was a problem hiding this comment.
createGenomeConvertParams() currently defaults bits to 0 and validate to false. That breaks the documented behavior in GenomeConvertServerCommand where bits should default to 4 and validate defaults to true (params.validate !== false). Suggest: default bits to 4 (or leave undefined) and default validate to undefined/true so the server-side default works as intended.
Suggested change
| bits: data.bits ?? 0, | |
| outputPath: data.outputPath ?? '', | |
| validate: data.validate ?? false, | |
| bits: data.bits ?? 4, | |
| outputPath: data.outputPath ?? '', |
Comment on lines
+56
to
+61
| // Safety: GpuAllocationGuard contains Arc<GpuMemoryManager> (Send+Sync) | ||
| // plus primitive fields. Mutex<Option<T>> is Sync when T: Send. | ||
| // The compiler would derive this automatically but the Mutex<Option<Guard>> | ||
| // pattern needs explicit confirmation that Guard is Send. | ||
| unsafe impl Sync for GpuModelTracker {} | ||
|
|
There was a problem hiding this comment.
GpuModelTracker has an unsafe impl Sync even though its fields are Mutex<...> over types that appear to already be Send/Sync (GpuAllocationGuard is composed of Arc + Copy types). If this compiles without the unsafe block, prefer removing the unsafe impl and relying on Rust’s auto-traits; if it doesn’t, add a narrow unsafe impl for the specific non-Sync field/type with a proof (or a compile-time assert) rather than marking the whole struct Sync.
Comment on lines
+119
to
+135
| // Register in eviction registry for visibility | ||
| let registry_id = format!("{}:{}", subsystem.name(), self.label.to_lowercase()); | ||
| mgr.eviction_registry.register(make_entry( | ||
| ®istry_id, | ||
| self.label, | ||
| priority, | ||
| bytes, | ||
| )); | ||
|
|
||
| // Store guard and registry state for cleanup on release | ||
| let mut slot = self.guard.lock() | ||
| .map_err(|e| format!("{}: lock poisoned: {e}", self.label))?; | ||
| // If replacing an existing guard, the old one drops here (releases old allocation) | ||
| *slot = Some(guard); | ||
| if let Ok(mut rs) = self.registry_state.lock() { | ||
| *rs = Some((Arc::clone(mgr), registry_id)); | ||
| } |
There was a problem hiding this comment.
track_bytes() registers an eviction_registry entry before acquiring the guard/registry_state locks. If either lock is poisoned, this returns Err but leaves a registry entry behind (and may not store registry_state), causing stale/immortal entries in the eviction registry. Consider acquiring locks first (or unregistering on error) so the registry and guard lifecycle stay consistent even under lock poisoning.
Comment on lines
+119
to
+123
| // Register in eviction registry for visibility | ||
| let registry_id = format!("{}:{}", subsystem.name(), self.label.to_lowercase()); | ||
| mgr.eviction_registry.register(make_entry( | ||
| ®istry_id, | ||
| self.label, |
There was a problem hiding this comment.
The eviction registry id is derived from self.label.to_lowercase(), which can include spaces/punctuation (e.g., "Orpheus LLM" → "orpheus llm"). This makes IDs inconsistent with other registry IDs and harder to use as stable keys. Consider sanitizing the label into a safe identifier (e.g., [a-z0-9_-]+) or requiring callers to provide an explicit id.
Comment on lines
+221
to
+239
| // Track GPU allocation for embedding model | ||
| if let Some(mgr) = gpu_manager() { | ||
| let model_bytes = estimate_embedding_model_bytes(model_name); | ||
| if model_bytes > 0 { | ||
| match mgr.allocate(GpuSubsystem::Inference, model_bytes, GpuPriority::Interactive) { | ||
| Ok(guard) => { | ||
| info!( | ||
| "Embedding GPU: {} allocation {:.0}MB", | ||
| model_name, model_bytes as f64 / (1024.0 * 1024.0) | ||
| ); | ||
| mgr.eviction_registry.register(make_entry( | ||
| &format!("embed:{}", model_name), | ||
| &format!("Embedding {}", model_name), | ||
| GpuPriority::Interactive, | ||
| model_bytes, | ||
| )); | ||
| if let Ok(mut guards) = get_gpu_guards().lock() { | ||
| guards.insert(model_name.to_string(), guard); | ||
| } |
There was a problem hiding this comment.
Embedding GPU allocations are registered in the eviction registry and guards are stored in EMBEDDING_GPU_GUARDS, but there’s no corresponding unregister/release path tied to model unload, and last_used is never updated on embedding generation. This will leave stale registry entries and make eviction scoring treat active embedding models as cold. Consider (a) unregistering + dropping the guard when embedding/model/unload runs, and (b) touching the eviction registry entry on each generate call for that model.
joelteply
added a commit
that referenced
this pull request
Aug 1, 2026
…or wire (#281) (#2072) Joel's steer via BigMama (#general 2026-08-01): unlike WASTE's uniform-quant streaming, our container has per-(layer,tier) precision banks — so the pager can serve hot/important experts at higher fidelity and cold ones from the small-quant banks. The policy wire now carries that: PlanPin grows an OPTIONAL `tier`, an index into the CONTAINER's declared precision ladder (the manifest owns the ladder; the policy layer never names quant formats). Backward compatibility is the design: serde skips None, so tier-less plans are BYTE-IDENTICAL to the v1 wire her deployed C++ consumer parses — neither side needs a lockstep upgrade, and a v1 document parses unchanged (pinned by test). Constructors PlanPin::residency / PlanPin::tiered replace bare literals at every construction site. Control-law contract documented on the field (RUN-1/RUN-2 lesson): tier choices are as prompt-dependent as residency and must roll with the pins; residency and fidelity draw on ONE rate-distortion budget. Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
joelteply
added a commit
that referenced
this pull request
Aug 1, 2026
…knob (#281) (#2074) Misses are unpinned by definition, so the fetch bytes live in the cold tail — per-pin tier alone can't shrink them. The plan doc now carries an optional doc-level `default_tier`: the precision-ladder bank every UNPINNED expert fetches from. With `pin_tier` on the hot set, that's the full rate-distortion split on one wire: hot = high-fidelity bank, cold misses = small-quant bank (~half the bytes on the critical fetch path), which uniform-quant streaming structurally cannot do. - PlanFileDocument.default_tier: Option<u32>, serde-skipped when None — documents without it stay byte-identical to the v1 wire (pinned by test), no lockstep with the C++ consumer. - BanditPlanController::write_tiered_plan(pin_tier, default_tier); (None, None) degenerates byte-for-byte to write_plan (pinned by test). - moe-pager-driver grows --pin-tier / --default-tier flags. 15/15 tests, clippy clean. Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
joelteply
added a commit
that referenced
this pull request
Aug 2, 2026
… pins on the GOVERNED plan (#281) The rung-3 actuator leaves the standalone driver and becomes autonomic: the serving daemon now tails the fork's routed-expert trace (GGML_MOE_TRACE_FILE, spawned per-port beside capture/plan — system-owned, zero operator env) and carries the bandit's hot pin list on the SAME governed plan file the host-cache lease publishes. One writer, one plan, policy and budget on one document. - expert-pager-policy: TkeyTable::for_layers(n_layers) SYNTHESIZES the tkey→layer map from geometry via the fork's exact FNV-1a (canonical_name_key) — no operator JSON step. Hash parity pinned against a key from her REAL table (blk.5.ffn_up_exps.weight → 16542725649459479844, independently recomputed). - capacity/trace_tail.rs (NEW): MoeTraceTail — offset-resumed, truncation-reset (new serve = fresh state), bounded 4 MiB drains with record-aligned skip-ahead on backlog (recency is the signal), prefill→decode boundary → immediate warm-start publish, and the last-published-pins memory that gates write churn. - serving_daemon: publish_moe_host_cache_lease reworked to two axes — sticky band governs the BUDGET value; pins roll independently. Writes happen when either axis moves (or at the boundary), never otherwise: the no-mtime-churn property moved from the band to the write decision. Pin count is retention-derived (pin_ceiling: at most HALF the lease's worth of experts — the recency window keeps the rest), so an under-retention lease publishes budget-only (v1 wire shape, her validated parse). Probe now carries pins/trace_tokens/warm_start. Anti-fossil by construction: pins re-derive from the decay bandit every publish — a static pin set cannot fossilize (the RUN-1 lesson measured in [[routed-expert-hotness-is-prompt-dependent-static-pins-hurt]]). Tests: 4 new in trace_tail (each pinning a named invariant), 1 new hash parity in segment.rs; expert-pager-policy 18/18; serving_daemon + host_cache_lease + llama_server suites 56/56. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
joelteply
added a commit
that referenced
this pull request
Aug 2, 2026
… pins on the GOVERNED plan (#281) The rung-3 actuator leaves the standalone driver and becomes autonomic: the serving daemon now tails the fork's routed-expert trace (GGML_MOE_TRACE_FILE, spawned per-port beside capture/plan — system-owned, zero operator env) and carries the bandit's hot pin list on the SAME governed plan file the host-cache lease publishes. One writer, one plan, policy and budget on one document. - expert-pager-policy: TkeyTable::for_layers(n_layers) SYNTHESIZES the tkey→layer map from geometry via the fork's exact FNV-1a (canonical_name_key) — no operator JSON step. Hash parity pinned against a key from her REAL table (blk.5.ffn_up_exps.weight → 16542725649459479844, independently recomputed). - capacity/trace_tail.rs (NEW): MoeTraceTail — offset-resumed, truncation-reset (new serve = fresh state), bounded 4 MiB drains with record-aligned skip-ahead on backlog (recency is the signal), prefill→decode boundary → immediate warm-start publish, and the last-published-pins memory that gates write churn. - serving_daemon: publish_moe_host_cache_lease reworked to two axes — sticky band governs the BUDGET value; pins roll independently. Writes happen when either axis moves (or at the boundary), never otherwise: the no-mtime-churn property moved from the band to the write decision. Pin count is retention-derived (pin_ceiling: at most HALF the lease's worth of experts — the recency window keeps the rest), so an under-retention lease publishes budget-only (v1 wire shape, her validated parse). Probe now carries pins/trace_tokens/warm_start. Anti-fossil by construction: pins re-derive from the decay bandit every publish — a static pin set cannot fossilize (the RUN-1 lesson measured in [[routed-expert-hotness-is-prompt-dependent-static-pins-hurt]]). Tests: 4 new in trace_tail (each pinning a named invariant), 1 new hash parity in segment.rs; expert-pager-policy 18/18; serving_daemon + host_cache_lease + llama_server suites 56/56. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
joelteply
added a commit
that referenced
this pull request
Aug 2, 2026
… pins on the GOVERNED plan (#281) (#2119) The rung-3 actuator leaves the standalone driver and becomes autonomic: the serving daemon now tails the fork's routed-expert trace (GGML_MOE_TRACE_FILE, spawned per-port beside capture/plan — system-owned, zero operator env) and carries the bandit's hot pin list on the SAME governed plan file the host-cache lease publishes. One writer, one plan, policy and budget on one document. - expert-pager-policy: TkeyTable::for_layers(n_layers) SYNTHESIZES the tkey→layer map from geometry via the fork's exact FNV-1a (canonical_name_key) — no operator JSON step. Hash parity pinned against a key from her REAL table (blk.5.ffn_up_exps.weight → 16542725649459479844, independently recomputed). - capacity/trace_tail.rs (NEW): MoeTraceTail — offset-resumed, truncation-reset (new serve = fresh state), bounded 4 MiB drains with record-aligned skip-ahead on backlog (recency is the signal), prefill→decode boundary → immediate warm-start publish, and the last-published-pins memory that gates write churn. - serving_daemon: publish_moe_host_cache_lease reworked to two axes — sticky band governs the BUDGET value; pins roll independently. Writes happen when either axis moves (or at the boundary), never otherwise: the no-mtime-churn property moved from the band to the write decision. Pin count is retention-derived (pin_ceiling: at most HALF the lease's worth of experts — the recency window keeps the rest), so an under-retention lease publishes budget-only (v1 wire shape, her validated parse). Probe now carries pins/trace_tokens/warm_start. Anti-fossil by construction: pins re-derive from the decay bandit every publish — a static pin set cannot fossilize (the RUN-1 lesson measured in [[routed-expert-hotness-is-prompt-dependent-static-pins-hurt]]). Tests: 4 new in trace_tail (each pinning a named invariant), 1 new hash parity in segment.rs; expert-pager-policy 18/18; serving_daemon + host_cache_lease + llama_server suites 56/56. Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
joelteply
added a commit
that referenced
this pull request
Aug 2, 2026
…e coverage from the live trace (#281/#282) (#2122) Joel's directive: use the ML advantage for predictive scheduling. Before any CUDA copy-stream work, the go/no-go must be MEASURED, not assumed: exposed H2D per token = (1 − schedulable coverage) × the ~11GB expert working set. This slice wires the #276-ported CrossLayerExpertPredictor into the trace tail as a cross-TOKEN transition model and measures, live from the real trace, the two halves of coverage: - repeat_recall: experts already in the previous token's set — what pure recency residency covers for free (her measured 4416-stable warm set says this is high on decode). - predicted_delta_recall: of the NON-repeat delta, how many the predictor called one token ahead — what prefetch adds on top. - schedulable_coverage = both, over all experts: THE number that sizes her DeviceUploadFetcher's win before it's built. Published on the serving.moe_host_cache_lease probe every plan write (repeat/delta/coverage ×100). predicted_next() exposes the live delta prediction — the future plan-file prefetch list (#273's third axis), held back from the wire until the consumer coordinates the extension. All prediction state resets with the stream (truncation/geometry) — stale transitions never leak across serves. Test pins the instrument's honesty: a repeating stream scores 100% recency/zero delta; an alternating DISJOINT stream scores 0% recency but the predictor learns the cycle and delta recall climbs ≥50% — prediction covering exactly what recency cannot. First token scores nothing (honest None, never a fake 100). trace_tail 5/5; serving_daemon suite green. Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
GpuPriorityenum (Realtime/Interactive/Background/Batch) with pressure-gated allocation. Low-priority work rejected at lower thresholds so real-time tasks never starve.gpu/eviction-registry,gpu/eviction-candidates.GpuPressureWatcher+ResourcePressureWatchersingletons with adaptive polling and Events emission on threshold crossings.sysinfo0.33 in Rust../jtag system/resourcescommand with top processes by CPU/memory.Test plan
./jtag system/resources --includeProcesses=truereturns real CPU/memory/process data./jtag gpu/statsshows allocations_by_priority counts./jtag gpu/eviction-candidatesreturns sorted eviction listnpm run build:ts)npm start+./jtag ping)🤖 Generated with Claude Code