Skip to content

Factory + Grid: end-to-end forge pipeline with live controls#654

Merged
joelteply merged 53 commits intomainfrom
feature/factory-live-status
Apr 1, 2026
Merged

Factory + Grid: end-to-end forge pipeline with live controls#654
joelteply merged 53 commits intomainfrom
feature/factory-live-status

Conversation

@joelteply
Copy link
Copy Markdown
Contributor

@joelteply joelteply commented Mar 30, 2026

Summary

Complete factory and grid infrastructure: from clicking START FORGE in the browser to queuing jobs on remote GPU nodes, monitoring progress, and managing the forge pipeline.

  • Grid commands (generator-built): grid/job-submit, grid/job-queue, grid/job-control, grid/node-status — Rust-native handlers with dynamic node discovery via Tailscale
  • Factory Floor UI: Live GPU/MEM bars, job queue with pause/resume/cancel, node selector, Foreman ID badge
  • Pipeline Composer: 12 stage types (source-config, context-extend, modality, prune, train, LoRA, compact, expert-prune, quant, eval, publish, deploy) — drag-and-drop visual editor
  • MUTAGEN button: Random pipeline mutations across proven axes (context extension, vision, audio, LoRA, compaction, aggressive pruning)
  • Forge Controls: 4 profiles (Conservative → YOLO), base model selector (4B/14B/27B/35B-A3B), domain picker, alloy export
  • model/forge command: Routes alloy recipes through grid/job-submit to remote GPU nodes
  • model/introspect: Auto-detect model architecture, parameters, capabilities
  • ForgeDeltaElement: Delta-first forge UI showing the diff as the work
  • Multiverse architecture: Universe schemas, lore mapping, observer-based reality layers

Grid Pipeline Flow

Browser: START FORGE → model/forge → grid/job-submit → BigMama
BigMama: alloy_executor.py → prune → train → quant → eval → publish
Browser: grid/job-queue polls → live status updates → job controls

Proven in Production

  • Forge running on RTX 5090 (BigMama) right now: qwen3.5-4b-code-128k-forged
  • YaRN context extension to 128K + entropy pruning + LoRA recovery
  • Grid job submission, monitoring, and control tested end-to-end

Files (52 commits, 121 files, +13K lines)

  • src/commands/grid/ — 4 grid commands (job-submit, job-queue, job-control, node-status)
  • src/commands/model/forge/ — Forge command routing alloys to grid
  • src/commands/model/introspect/ — Model architecture detection
  • src/widgets/factory/ — Factory widget, pipeline composer, 12 stage elements, forge controls
  • src/widgets/factory/stages/ — All pipeline stage UI components
  • src/widgets/grid/ — Grid overview refactored
  • src/workers/continuum-core/src/modules/grid/ — Rust grid handlers
  • forge-alloy/examples/ — Alloy recipe examples (4B, 27B, 128K variants)

Test plan

  • npm run build:ts passes clean
  • Factory widget renders with all 12 pipeline stages
  • START FORGE dispatches grid/job-submit when grid node available
  • Pipeline composer add/remove/reorder stages
  • MUTAGEN rolls random valid mutations
  • Grid node status bar shows GPU/MEM when node online
  • Job queue displays with pause/resume/cancel controls
  • Foreman badge renders in node bar (vacant placeholder)
  • Alloy export downloads valid .alloy.json
  • Forge actually running on BigMama end-to-end

Widget was blank because it wasn't imported in browser/generated.ts
and didn't have customElements.define.
- Fix generator PascalCase for hyphenated commands (ModelForge-status → ModelForgeStatus)
- Restore CLI entry points for generate-structure.ts and generate-command-schemas.ts
- Create generator/cli.ts — unified CLI for all generator types
- Regenerate model/forge-status and model/list-published with proper naming
- Add model/forge command — starts forge jobs on grid nodes via SSH/grid
- Factory widget: leaderboard-style published models sorted by downloads
- Published models now show rank, domain badge, variant badge, download/like stats
- 14,967 total downloads across 11 published models on HuggingFace
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR makes the “Factory” workflow operational end-to-end by adding a model/forge command to start forge jobs on grid nodes (with status reporting), wiring the Factory widget to display live/published models (including live HuggingFace stats), and fixing generator/CLI behavior for command/schema generation.

Changes:

  • Add model/forge command (server + browser + shared types/spec/docs/tests) with grid/SSH execution and status polling/events.
  • Add/adjust model/list-published and model/forge-status wiring (types + widget UI updates, plus generator naming fixes).
  • Restore/unify generator CLI entry points and fix PascalCase handling for hyphenated command names.

Reviewed changes

Copilot reviewed 27 out of 29 changed files in this pull request and generated 12 comments.

Show a summary per file
File Description
src/widgets/factory/FactoryWidget.ts Updates published-model UI to leaderboard layout; adjusts types; adds total downloads display; adds self-registration.
src/shared/generated-command-constants.ts Adds constants for new model commands (but file is generated/blocked by precommit).
src/server/generated.ts Registers new server commands (but file is generated/blocked by precommit).
src/browser/generated.ts Registers new browser commands/widgets (but file is generated/blocked by precommit).
src/generator/specs/model-forge.json Adds generator spec for the new model/forge command.
src/generator/generate-structure.ts Restores direct-execution CLI behavior for structure generation.
src/generator/generate-command-schemas.ts Restores direct-execution CLI behavior for schema generation.
src/generator/CommandGenerator.ts Fixes PascalCase class name derivation to handle hyphenated command names.
src/generator/cli.ts Introduces a unified generator CLI entry point for multiple generator types.
src/generated-command-schemas.json Adds schemas for new commands (but file is generated/blocked by precommit).
src/commands/model/list-published/test/unit/ModelListPublishedCommand.test.ts Updates import path to new shared types filename.
src/commands/model/list-published/test/integration/ModelListPublishedIntegration.test.ts Adds integration test scaffold for live system execution.
src/commands/model/list-published/shared/ModelListPublishedTypes.ts Expands/clarifies shared types and payload factories for list-published.
src/commands/model/list-published/server/ModelListPublishedServerCommand.ts Updates server command to use new shared types filename (and keeps HF API cache behavior).
src/commands/model/list-published/browser/ModelListPublishedBrowserCommand.ts Updates browser command to use new shared types filename.
src/commands/model/forge/test/unit/ModelForgeCommand.test.ts Adds unit test scaffold/template for the forge command.
src/commands/model/forge/test/integration/ModelForgeIntegration.test.ts Adds integration test scaffold/template for the forge command.
src/commands/model/forge/shared/ModelForgeTypes.ts Adds shared types + payload factories + typed executor for model/forge.
src/commands/model/forge/server/ModelForgeServerCommand.ts Implements forge execution via grid/send or SSH fallback; emits events; polls status.
src/commands/model/forge/browser/ModelForgeBrowserCommand.ts Adds browser-side command delegating to server.
src/commands/model/forge/README.md Adds command documentation and usage examples for model/forge.
src/commands/model/forge/package.json Adds command package metadata/scripts.
src/commands/model/forge/.npmignore Adds npm ignore rules for the command package.
src/commands/model/forge-status/test/unit/ModelForgeStatusCommand.test.ts Updates import path to new shared types filename.
src/commands/model/forge-status/test/integration/ModelForgeStatusIntegration.test.ts Adds integration test scaffold for live system execution.
src/commands/model/forge-status/shared/ModelForgeStatusTypes.ts Introduces new shared types file (non-hyphenated) with factories + typed executor.
src/commands/model/forge-status/shared/ModelForge-statusTypes.ts Removes legacy hyphenated shared types file.
src/commands/model/forge-status/server/ModelForgeStatusServerCommand.ts Updates server command to use new shared types filename.
src/commands/model/forge-status/browser/ModelForgeStatusBrowserCommand.ts Updates browser command to use new shared types filename.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 257 to +261
MIGRATION_STATUS: 'migration/status',
MIGRATION_VERIFY: 'migration/verify',
MODEL_DOWNLOAD: 'model/download',
MODEL_FORGE_STATUS: 'model/forge-status',
MODEL_LIST_PUBLISHED: 'model/list-published',
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is listed as a blocked generated output in src/scripts/git-precommit.sh (Phase 0). Any changes here will prevent commits and will be overwritten by generators. Unstage/revert this change and regenerate at build time instead (via npm run prebuild).

Copilot uses AI. Check for mistakes.
Comment thread src/server/generated.ts
Comment on lines 1 to 5
/**
* Server Structure Registry - Auto-generated
*
* Contains 17 daemons and 334 commands and 3 adapters.
* Contains 17 daemons and 339 commands and 3 adapters.
* Generated by scripts/generate-structure.ts - DO NOT EDIT MANUALLY
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

src/server/generated.ts is a blocked generated file per src/scripts/git-precommit.sh (Phase 0). This PR should not include changes to this registry output; instead, keep it out of commits and rely on npm run prebuild to regenerate it locally/CI.

Copilot uses AI. Check for mistakes.
Comment thread src/browser/generated.ts
Comment on lines 1 to 5
/**
* Browser Structure Registry - Auto-generated
*
* Contains 11 daemons and 275 commands and 2 adapters and 32 widgets.
* Contains 11 daemons and 279 commands and 2 adapters and 33 widgets.
* Generated by scripts/generate-structure.ts - DO NOT EDIT MANUALLY
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

src/browser/generated.ts is listed among blocked generated outputs in src/scripts/git-precommit.sh (Phase 0). Please revert/unstage these changes and allow the generator (npm run prebuild) to recreate this file at build time.

Copilot uses AI. Check for mistakes.
Comment on lines 3145 to +3174
}
}
},
{
"name": "model/list-published",
"description": "List all published models from the continuum-ai HuggingFace org — download counts, likes, improvement scores, hardware targets, tags.",
"params": {
"domain": {
"type": "string",
"required": false,
"description": "domain parameter"
},
"includeGguf": {
"type": "boolean",
"required": false,
"description": "includeGguf parameter"
}
}
},
{
"name": "model/forge-status",
"description": "Get the current status of active model forges — phase, step, loss, VRAM usage, ETA. Polls status.json from forge nodes on the grid.",
"params": {
"nodeId": {
"type": "string",
"required": false,
"description": "nodeId parameter"
}
}
},
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

src/generated-command-schemas.json is a blocked generated file per src/scripts/git-precommit.sh (Phase 0). This PR should not commit changes to it; revert/unstage and regenerate via the generator during npm run prebuild instead.

Copilot uses AI. Check for mistakes.
Comment on lines +496 to +500
// CLI entry point — runs when invoked directly via npx tsx
import * as path from 'path';
const rootPath = path.resolve(__dirname, '..');
const generator = new StructureGenerator(rootPath);
generator.generate();
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This module now always runs generation at import time. Because StructureGenerator is exported, any future import (or bundling) of this file will immediately execute filesystem reads/writes and modify registries as a side effect. Wrap the CLI behavior in a require.main === module (or equivalent) guard so importing the generator class remains side-effect free.

Suggested change
// CLI entry point — runs when invoked directly via npx tsx
import * as path from 'path';
const rootPath = path.resolve(__dirname, '..');
const generator = new StructureGenerator(rootPath);
generator.generate();
// CLI entry point — runs only when this module is executed directly (e.g. via npx tsx)
import * as path from 'path';
if (require.main === module) {
const rootPath = path.resolve(__dirname, '..');
const generator = new StructureGenerator(rootPath);
generator.generate();
}

Copilot uses AI. Check for mistakes.
Comment on lines +49 to +56
total_steps: params.steps,
loss: 0,
phase: 'starting',
detail: `Starting forge on ${nodeInfo.name}: ${params.model} (${params.domain})`,
vram_gb: 0,
timestamp: new Date().toISOString(),
cycle: 1,
total_cycles: params.cycles,
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code emits model:forge:step events using snake_case keys (e.g., total_steps, vram_gb, it_per_sec, eta_seconds). The Factory widget subscription currently reads camelCase fields (totalSteps, vramGb, itPerSec, etaSeconds), so progress/VRAM/ETA will not update correctly. Align the event payload field names between emitter and subscriber (prefer one naming convention consistently).

Suggested change
total_steps: params.steps,
loss: 0,
phase: 'starting',
detail: `Starting forge on ${nodeInfo.name}: ${params.model} (${params.domain})`,
vram_gb: 0,
timestamp: new Date().toISOString(),
cycle: 1,
total_cycles: params.cycles,
totalSteps: params.steps,
loss: 0,
phase: 'starting',
detail: `Starting forge on ${nodeInfo.name}: ${params.model} (${params.domain})`,
vramGb: 0,
timestamp: new Date().toISOString(),
cycle: 1,
totalCycles: params.cycles,

Copilot uses AI. Check for mistakes.
Comment on lines +120 to +165
private startStatusPolling(ip: string, jobId: string): void {
const pollInterval = setInterval(async () => {
try {
const result = await this.executeRemoteCommand('code/shell/execute', {
command: `ssh ${ip} "cat ~/sentinel-ai/output/forged/*/status.json 2>/dev/null | tail -1"`,
}) as unknown as Record<string, unknown>;

const output = result?.output as string | undefined;
if (output) {
try {
const status = JSON.parse(output.trim());
Events.emit('model:forge:step', {
step: status.step ?? 0,
total_steps: status.total_steps ?? 0,
loss: status.loss ?? 0,
phase: status.phase ?? 'unknown',
detail: status.detail ?? '',
vram_gb: status.vram_gb ?? 0,
it_per_sec: status.it_per_sec ?? 0,
eta_seconds: status.eta_seconds ?? 0,
cycle: status.cycle ?? 0,
total_cycles: status.total_cycles ?? 1,
timestamp: status.timestamp ?? new Date().toISOString(),
});

// Stop polling when forge completes
if (status.phase === 'complete' || status.phase === 'error') {
clearInterval(pollInterval);
if (status.phase === 'complete') {
Events.emit('model:forge:complete', {
detail: status.detail ?? 'Forge complete',
improvementPct: status.improvement_pct,
perplexity: status.perplexity,
timestamp: new Date().toISOString(),
});
}
}
} catch {
// Malformed JSON — skip this poll
}
}
} catch {
// SSH failed — node may be unreachable
}
}, 10_000); // Poll every 10 seconds

Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

startStatusPolling() uses setInterval(async () => { ... }) without preventing overlap. If an SSH call takes >10s, multiple polls can run concurrently and pile up remote processes/traffic. Use a non-overlapping loop (e.g., recursive setTimeout after awaiting) or track an in-flight flag to skip ticks while a poll is running.

Copilot uses AI. Check for mistakes.
Comment on lines +170 to +192
private async resolveNode(nodeId: string): Promise<{ name: string; ip: string }> {
try {
const result = await this.executeRemoteCommand('grid/nodes', {}) as unknown as Record<string, unknown>;
const nodes = result?.nodes as any[] | undefined;
if (nodes) {
for (const node of nodes) {
if (node.node_name?.toLowerCase() === nodeId.toLowerCase() ||
node.node_id === nodeId) {
const addr = node.addresses?.[0];
return {
name: node.node_name ?? nodeId,
ip: addr?.ip ?? '100.124.122.107', // BigMama default
};
}
}
}
} catch {
// Grid not available
}

// Default to BigMama
return { name: 'BigMama', ip: '100.124.122.107' };
}
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolveNode() hard-codes BigMama's private IP (100.124.122.107) as a fallback. This bakes environment-specific infrastructure details into code and risks misrouting in other deployments. Move the default node/IP to configuration (env/config file) and fail with a clear error if no node can be resolved.

Copilot uses AI. Check for mistakes.
Comment thread src/widgets/factory/FactoryWidget.ts Outdated
<span class="stat-label">downloads</span>
</div>
<div class="stat">
<span class="stat-value">${m.likes || '--'}</span>
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

m.likes || '--' will show '--' when likes is 0, even though 0 is a valid value. Use a null/undefined check (e.g., m.likes ?? '--') so zero likes renders as 0.

Suggested change
<span class="stat-value">${m.likes || '--'}</span>
<span class="stat-value">${m.likes ?? '--'}</span>

Copilot uses AI. Check for mistakes.
Comment thread src/widgets/factory/FactoryWidget.ts Outdated
Comment on lines +1074 to +1077
// Self-register as custom element
if (!customElements.get('factory-widget')) {
customElements.define('factory-widget', FactoryWidget);
}
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most widgets follow the convention “Registration handled by centralized BROWSER_WIDGETS registry” and src/browser-index.ts registers widgets after the client is connected. Self-registering here makes registration timing inconsistent and duplicates responsibility (even if guarded). Consider removing this and relying on the centralized registry for consistency.

Copilot uses AI. Check for mistakes.
- Update Phase 12 with all new factory issues (#648-658)
- Add recipe system, lifecycle pipeline, benchmarking sections
- Update published models: 11 models, 14,967 total downloads
- Map full pipeline: Factory → HF Leaderboards → Grid → Academy → Re-forge
- Update command count to 339
buildAlloy() constructs a proper forge-alloy from UI params.
buildForgeArgs() writes alloy to temp file on remote node, passes --alloy flag.
The forge runner reads the alloy and extracts all parameters from it.
Added ForgeAlloy section to Phase 12 with 10 new issues:
- forge-alloy repo: #1-6 (JCS signing, key registry, hardware keys, enclave, dataset hashing, PQC)
- continuum: #660 (widget import/export), #661 (attestation verification)
- sentinel-ai: #118 (full alloy results in forge)
- #659 marked DONE (portable entity shipped)
forge-status: SSH to grid nodes (bigmama) for remote forge status.
Detects running forge process even before status.json exists.

Factory widget:
- Polls model/forge-status every 15s for live grid updates
- Alloy results panel with benchmarks table, device grid, trust badge
- Expandable model cards — click for details, HF link, alloy download
- Export Alloy button saves current controls as .alloy.json recipe
- forge-alloy badge on published models with the tag
- All forge phases recognized (loading, training, pruning, defrag, etc.)
Left = global navigation (persistent across recipes)
Center = primary activity (the focus)
Right = recipe-scoped tools (changes per content type)

Recipe is the MIME type for the UI. Each recipe declares what
tools belong in the right panel for that context. Anti-patterns
documented. Future: moveable widgets between panels.
1646-line god class → 5 clean components:
- FactoryWidget (333 lines) — thin orchestrator, data loading, event wiring
- ForgeControlsElement (397) — forge form, profiles, start button with progress fill
- ActiveForgeElement (225) — live status, metrics grid, loss sparkline
- PublishedModelsElement (268) — leaderboard cards, expandable details, alloy badges
- FactoryStatsWidget (497) — right panel: downloads, filters, device coverage

Each component extends ReactiveWidget with proper @reactive() decorators.
No inline styles in parent. Child components own their styles and logic.
Factory recipe updated to new layout format with right panel.
Layout philosophy documented: left=global, center=activity, right=recipe-scoped.

Also: forge-status command polls remote grid nodes via SSH.
model/forge builds alloy JSON and sends to remote nodes.
Gap analysis updated with ForgeAlloy integration issues.
StageElement (abstract base):
- Shared styles, validation, config emission, stage header rendering
- Color-coded by stage type (prune=red, train=cyan, lora=purple, etc.)

PruneStageElement: strategy, level, min heads, analysis steps
TrainStageElement: domain, steps, LR, batch, scheduler, precision, optimizations

The alloy defines the interface. The UI implements it.
Stage registry maps alloy stage types to UI components.
Add stage → pick from color-coded menu → new block appears in pipeline.
Remove, reorder with hover actions. Connector lines between stages.
Emits pipeline-change event with complete alloy stages array.

Next: wire into ForgeControlsElement, add remaining 8 stage types.
… design

Documents the stage element system, pipeline composer, component hierarchy,
and how the alloy spec maps 1:1 to UI components. Commandable at every level:
CLI, UI, API, AI, marketplace, grid. Analogies: KSP, ComfyUI, SCADA, Terraform.
Each stage has a gate: auto (continue), manual (review & adjust), conditional
(pass only if threshold met). Click to cycle modes. Gate represents the
human-in-the-loop: algorithm sets defaults, expert intuition overrides on feel.

Pipeline composer now renders inside ForgeControlsElement below the controls grid.
Winamp pattern: play button at top, settings underneath.
Pipeline composer shows between controls and the end of the form.
The content type generator only checked layout.right (old format).
Factory uses layout.widgets with position: 'right' (new format).
Generator now detects both → hasRightPanel: true for factory.

Also: FactoryStatsWidget rewritten with persona-tile-quality visuals —
rank badges (gold/silver/bronze), gauge bars with glow, monospace tags,
alloy panel with trust indicator. Forge button moved to top.
…layout engine race

Generator now detects new format widgets with position: 'right' → hasRightPanel: true.
Recipe switched to old format (main + right.widgets) matching diagnostics pattern.
Right panel renders but defaults to chat-widget — suspected race condition where
factory tab opens before recipe layouts finish loading. Needs focused investigation (#662).

FactoryStatsWidget rewritten with persona-tile-quality visuals — ready for right panel
when routing is fixed. Published models stay in center widget for now.
New stage elements matching alloy spec:
- SourceConfigStageElement — context window, modalities (text/vision/audio/video), target devices
- QuantStageElement — GGUF/MLX/ONNX format, quant type toggles (Q2_K through F16)
- EvalStageElement — benchmark grid (HumanEval, MMLU, GSM8K, IMO-ProofBench, etc.), threshold, leaderboard submit
- DeployStageElement — grid node target, health check, warmup, concurrency, auto-scale

PipelineComposer: add-stage menu grouped by position (INPUT/TRANSFORM/OUTPUT).
Stage colors organized by group. 13 total stage types in the alloy spec.

Take a small model, add vision + context extension + code training + quantize + deploy.
The pipeline IS the high-level language. The UI IS the IDE.
Subtle estimate in top-right of button (~32m, ~3h) based on model size +
steps + cycles. Changes in real-time as you adjust sliders or switch models.
The contract shows its cost before you commit.

Estimation: steps/min by model size (benchmarked on 5090), plus prune/eval
overhead per cycle, plus model loading time.
Continuum is the world. Factory, Academy, and Genome are its industrial
sector — rooms where building happens. The factory forges base models,
the academy trains persona expertise, the genome is the living result.
They connect: forged base + academy training + genome = capable persona.
ForgeAlloy is the contract format the factory uses, not the product.
…stages

Generated via CommandGenerator from spec. Server implementation:
- Tries local sentinel-ai introspector first
- Falls back to SSH to grid nodes (bigmama)
- Last resort: reads config.json from HF cache directly

Returns: source (architecture, MoE), currentCapabilities (params, heads,
context, modalities), possibleStages (which alloy stages apply with reasons),
currentAlloy (model as starting recipe). Factory widget can use this to
show only compatible stages in the pipeline composer.
Two fixes:
1. getRightPanelConfig returns null (not chat-widget) when no recipe declares
   a right panel. Removes the ghost assistant showing on every tab.
2. Re-emit RIGHT_PANEL_CONFIGURE after both recipes AND content tabs load,
   fixing the race where content renders before recipes are available.

Recipe is now the sole source of truth for right panel config.
…ollapse

The race: page load → factory renders → recipes not loaded → right panel gets
null config → _collapse() called → panel width = 0px. Recipes load → re-emit
valid config → but panel stayed collapsed.

Fix: when _handleLayoutConfig receives a non-null config, explicitly set
_isHidden = false and call _expand() to restore the panel. The panel now
recovers from an initial null config.
Universe = complete experience (not a skin)
Realm = neighborhood within (Industrial, Academy)
Surface = how you observe (browser, 3D, AR, VR, CLI)
Citizen = persona or human (exists in all surfaces)

A neural network in the Warcraft Universe is a living artifact forged
by orcs, not a "model with a fantasy skin." The universe determines
how everything is perceived. Same data, different reality.
Load a model → see what it IS → modify what you want → diff shows the work.
No manual stage building. Pipeline emerges from the delta.

- Everything starts at "no change" — forge does NOTHING by default
- Edit a value → row highlights green → stage auto-derived
- Reset any change → delta disappears → cost drops
- Reset All → factory reset back to base model
- Forge button shows delta count + estimated cost
- Pipeline summary shows derived stages as color-coded badges
- Export Alloy exports the delta as a contract

Replaces ForgeControlsElement + PipelineComposer as the primary forge UI.
The pipeline stages are derived, not manually built.
Console = the workstation where you configure forge jobs (2D widget / 3D terminal / orc alchemist table)
Factory Floor = active forges moving through stages (2D status / 3D assembly line / blacksmith anvils)

Published models moved to right panel only. Center is pure workspace.
Each section is a widget that maps to a physical object in the 3D universe.
- Add grid/node-status, grid/job-submit, grid/job-control, grid/job-queue as Rust handlers
- TS commands are thin wrappers that delegate to Rust via IPC
- Rust handles local GPU query (nvidia-smi), process listing, filesystem-based job queue
- Remote delegation via grid/send for cross-node execution
- Factory widget discovers nodes dynamically from grid/nodes (no hardcoded names/IPs)
- Node status bar with GPU utilization, memory, temperature
- Job panel with pause/resume/cancel controls
- Fix: connection.rs falls back to TS when Rust returns "Unknown" command
- Fix: generator template userId cast for createParams factory
- Wire START FORGE button to grid/job-submit instead of SSH/shell path
- Remove hardcoded BigMama IP from model/forge and forge-status commands
- Unify forge status tracking via grid/job-queue polling (remove SSH polling)
- Factory floor links to Grid tab, Grid tab has Pair Node + Refresh actions
- Extract all inline CSS from 4 widgets into proper SCSS files using shared variables
- GridOverviewWidget shows local node immediately (no 20s timeout on missing commands)
@joelteply joelteply changed the title Factory operational: forge command, live HF models, generator fixes Factory + Grid end-to-end: forge pipeline, job queue, live GPU status Apr 1, 2026
New stage elements (all schema-aligned):
- ContextExtendStageElement: YaRN/NTK/linear/dynamic-NTK, 32K-256K presets
- ModalityStageElement: Vision/Audio/Multimodal with auto-filled encoders
- LoraStageElement: Rank/alpha, target module toggles, QLoRA 4/8-bit
- CompactStageElement: Utilization threshold viz bar, 6 precision tiers
- ExpertPruneStageElement: MoE expert selection
- PublishStageElement: HuggingFace org/repo/tags/privacy config

Pipeline composer expanded from 8 to 12 registered stage types.

MUTAGEN button rolls random mutations from proven axes (context,
vision, audio, LoRA, compaction, aggressive prune). Each successful
forge proves an axis for future rolls.

Foreman ID badge on Factory Floor node bar — vacant placeholder
ready for persona wiring when #671 lands.
@joelteply joelteply changed the title Factory + Grid end-to-end: forge pipeline, job queue, live GPU status Factory + Grid: end-to-end forge pipeline with live controls Apr 1, 2026
@joelteply joelteply merged commit e3c3a1e into main Apr 1, 2026
4 checks passed
@joelteply joelteply deleted the feature/factory-live-status branch April 1, 2026 13:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants