Hardware Visualisation

Hardware Visualisation (Digital Twin)

The Hardware Visualisation screen (/hardware-twin) renders the saved automation topology read-only in 3D and overlays a live picture of equipment activity and handling-unit movement derived from the device-task feed — giving a real-time digital twin of the physical system without any extra backend instrumentation.

Backed by GET /api/flow/device-tasks (flow-orchestrator, port 8085), polled every 2 s. RBAC: ADMIN, SUPERVISOR, or OPERATOR role required.

Equipment state

The base scene is rendered using the same meshes as the Automation Topology Editor (two-tone conveyor ribbons, racked ASRS, sorter/workstation boxes, SCAN/QRY/DIV function-point markers and labels). Runtime state is shown as a clean overlay on top of the editor scene. The overlay type differs by equipment category.

Conveyor body colour

Conveyors wear their state as the colour of the belt body itself — the body material of the editor mesh is driven directly by the derived state (hardwaretwin/conveyorState.ts derives it, HardwareTwin3D.tsx feeds it through the bodyTint prop on EquipmentMesh/ConveyorPath, recomputed per snapshot plus a 1 s hysteresis tick). The color prop is intentionally bypassed for real conveyors because ConveyorPath hard-codes the body material when cat === 'conveyor'; bodyTint is the dedicated override path for the twin. There is no translucent overlay layer; the earlier skin overlay washed the blue body out and was removed.

Belt body colour	State	Trigger
Green	Functional	No fault; normal (or no) traffic on this segment. A subtle tint so a healthy floor stays readable.
Orange	Jam / heavy traffic	Any of: (a) a stalled tote — its scan timeline has not advanced for > 2.5× the expected hop time; (b) tote density ≥ 0.5 totes/m of belt path length (min 2 totes); or (c) a HELD divert decision on a tote currently riding this belt. A 4 s hysteresis holds the orange state after the raw signal clears so borderline polls do not strobe the skin.
Red	Fault	An active device task on this equipment reported FAILED. Red takes priority over orange over green.

Attribution — which belt a tote "rides", for the jam signal — uses buildBeltLocator from motion.ts (a nearest-belt projection of the tote's current position). This projection is used ONLY for belt-state attribution; tote motion no longer projects onto belts (it plays the backend-resolved node polyline — see Live totes below).

Non-conveyor equipment overlays

Non-conveyor equipment (ASRS racks, workstations, sorters without a skinnable belt body) retains the floating activity sphere overlay:

State	Overlay	Trigger
Idle	None — base mesh only	No active task seen in the current window
Running	Floating amber sphere above the equipment, pulsing in size and brightness	At least one active (REQUESTED/DISPATCHED) task on this equipment
Faulted	Floating red sphere above the equipment (steady)	Any FAILED task on this equipment in the current window

Selected equipment of any type shows a lime floor ring below it.

Tote (handling-unit) markers

The twin shows two distinct layers of handling-unit markers that together give a complete inventory picture of the warehouse at a glance.

Live totes (in-transit, queued, done)

Active handling units are drawn as 3D tote models (600 × 400 × 320 mm open-top container — base plate, four slightly tapered walls, and a bright state-coloured rim) whose positions replay the observed scan trail — no motion is invented.

In-transit positioning — the backend "visu master" (Shape B): tote motion is supplied by the backend, not reconstructed on the client. The flow-orchestrator owns both the routing graph and the scan trace, so GET /api/flow/twin/tote-paths?warehouseId= (TwinPathService) returns, per in-transit HU, its actual traversed-node polyline — each node's world XZ position (from conveyor_node, the same coordinates the 3D scene renders), in the order it was scanned on the running CONVEY leg, with each scan's timestamp and the routed-to lead node — plus the server clock. The twin polls this once per tick and feeds the waypoints into the per-tote interpolation buffer (ui/src/hardwaretwin/motion.ts). The 3D layer renders each tote at serverNow − 5 s: this delay is deliberate — with the 2 s poll the render point sits ≈2–3 scans behind the newest data, so totes are essentially always interpolating between two known scanned positions rather than dead-reckoning (operator preference: a few seconds of latency is acceptable; incorrect predicted motion is not). The render clock is anchored to the server clock returned in the payload, so client/server skew can never push sampling outside the buffered window. Consecutive waypoints are graph-adjacent nodes, so the layer connects them with straight segments — which are the belt sections — and interpolates by time. On a buffer underrun it dead-reckons gently toward the routed-to lead node and eases to a stop; revised data blends in over ~0.5 s; genuine discontinuities (rack store/retrieve) teleport.

Why this replaced client-side reconstruction. The earlier design fetched the raw trace and, to draw motion between two scans, projected each scanned position onto the nearest drawn belt to recover "how far along" it was. At a divert two branches meet at one point, so that projection is ambiguous: it landed near a belt's start and drew the path from the divert back to the start and forward again — totes flew to the conveyor's beginning and snapped back every poll. The backend has no such ambiguity (it logged exactly which edge it routed), so handing the frontend the resolved node sequence removes the entire guess-the-belt bug class, and the server clock removes the skew class. The frontend no longer projects onto belts for tote motion at all.

Queued totes: the queue state is sourced from the real induction queue per GTP workstation (flow's getStationQueue API), so a tote that has finished all work simply disappears rather than lingering forever as "queued" from a stale completed CONVEY task. The twin falls back to task-inference when queue data is unavailable.

Queued totes are positioned on the station's inbound conveyor, not inside the workstation box. The 3D layer reads the editor's explicit node-link between the station and its inbound belt to find the link point (the fromPathIndex / toPathIndex of the connection, falling back to a projection of the station centre onto the belt, chosen by geometry — the belt whose directed sections arrive at the link node, so an outfeed belt is never picked). The queue head (queueIndex 0) sits at the link point — held there until the workplace releases it, matching reality (a GTP tote dwells on the conveyor at the pick position and never moves onto the workstation box); slot i sits i × 0.8 m upstream — so the queue is a visible line of totes on the inbound belt with the active tote at the front. When queue data comes from the live API, queueIndex is the actual induction-queue position; in the task-inference fallback it is the arrival-order counter per station (inferredQueueCount).

Anti-stack for simultaneously-retrieved totes: two totes retrieved from the ASRS together share a near-identical backend path, so they would otherwise render on the same spot and travel as a single merged tote. A per-frame anti-stack pass runs after target positions are resolved: if two moving totes are heading the same direction (dot-product ≥ 0.7) and sit closer than 0.8 m nose-to-tail, the trailing tote is pushed back along its own heading until they reach minimum separation. The pass runs in two relaxation sweeps (SEP_PASSES = 2) to resolve short chains. Because the correction uses each tote's own travel direction — never a belt re-projection — it cannot snap a tote to a crossing belt's start at a divert (headings diverge at diverts and the pass disengages). Only moving in-transit totes participate; queued and stored totes are excluded.

Equipment-activity anchoring still uses a family-level fallback (ASRS/AUTOSTORE/AMR → storage placement; CONVEYOR → longest-path conveyor) but only for the amber/red activity sphere overlay, never for tote-marker positions.

Totes at non-conveyor equipment (workstations, ASRS placements) sit on the top surface of that equipment's bounding box so they are visibly above the solid mesh.

Tote state	Colour	Meaning
In-transit	Bright state-coloured rim	Tote has an active CONVEY task; position follows the scanned node trail
Recirculating	Bright state-coloured rim	Last sorter decision was RECIRCULATED or HELD; loops back to induct
Queued	Bright state-coloured rim	Tote is in the real induction queue at a GTP workstation
Done	Bright state-coloured rim	Tasks completed; marker lingers briefly then is removed

Stored totes — rack contents (ADR-0009 §5)

HUs that are at rest in ASRS storage are shown as bright ice-blue mini-totes rendered directly inside the rack mesh at their exact cell position. This is a pure representation of the live HU registry (populated via transport lifecycle — see Inbound and Inventory); no position is invented or guessed:

• Cell X spreads the tote along the rack's length.
• Shuttle level (posY) stacks totes up the rack height.
• Side (L/R) places the tote in the correct rack flank relative to the aisle.
• Channel depth (posZ) pushes the tote outward from the aisle face.

HUs whose master-data location lacks cell coordinates (posX/posY) are skipped entirely. HUs that already appear as live totes (above) are excluded to prevent duplicates.

Clicking a stored-tote mini opens the same HU detail / trace panel as clicking a live tote. When selected, a small label showing the HU code floats above the mini.

Master-data locations (cell coordinates) are loaded once per warehouse alongside the topology; the HU registry is re-fetched on every poll tick and deduplicated against the live-tote set.

Stats bar

A bar across the top of the screen shows live aggregate counters derived from the current device-task window:

Counter	Description
In-transit	Totes with at least one active transport task
Queued	Totes queued at a workstation
Throughput/min	Completed tasks in the last 60 s
Recirculations	Sorter RECIRCULATED decisions in the current window
Faults	Equipment with at least one FAILED task
In storage	Totes currently at rest in ASRS rack cells (live HU registry)

Level filter

When the topology spans multiple floor levels, a level selector in the header restricts the scene to a single level. All unrelated equipment and tote markers are hidden; the stats bar counts only the visible level.

Labels toggle

A Labels checkbox in the control bar (default off) shows or hides the text labels that float above each equipment placement and function-point marker:

Setting	Effect
Off (default)	All decorative overlays are hidden: equipment-code text, SCAN/QRY/DIV function-point cones/diamonds, and conveyor junction rings. Only the base 3D meshes, tote markers, and activity/state overlays remain. Maximum signal-to-noise at a glance.
On	Equipment codes appear above each placement; SCAN/QRY/DIV labels and cone/diamond glyphs appear above each function-point; conveyor junction rings are shown. Matches the Automation Topology Editor view.

The toggle does not affect the 3D geometry, tote markers, or activity/state overlay indicators.

Interactions

Action	Result
Click equipment	Opens a detail panel with the equipment's recent device tasks
Click tote marker	Opens the HU trace panel for that handling unit (same trace as Transport Overview#per-hu-transport-trace)
Click background / grid	Clears both selections
Left-drag	Orbit the camera
Right-drag	Pan
Scroll / middle-drag	Zoom

Implementation notes

• Data sources per poll tick:

  Source	Endpoint	Purpose
  Device tasks	GET /api/flow/device-tasks	Equipment activity + tote state
  Live-twin tote paths	GET /api/flow/twin/tote-paths?warehouseId=	Backend-resolved traversed-node polylines + server clock → in-transit tote positions
  Station queue	GET /api/flow/gtp/queue?stationId= (per workstation)	Real queued-tote truth source
  Handling units	GET /api/inventory/handling-units	Full HU registry → stored-tote positions inside the rack

• Data sources loaded once per warehouse (on topology load):

  Source	Endpoint	Purpose
  Conveyor topology	GET /api/flow/conveyor/topology?warehouseId=	Routing-node world positions for scan-replay
  Master-data locations	GET /api/masterdata/locations?warehouseId=	Cell coordinates (posX, posY, posZ, side) used by deriveStoredTotes

• Coordinate system: all twin geometry is floor-relative — matching the Automation Topology Editor, which renders each level on the floor plane regardless of the level's elevationM. The PlacementGeom.baseY field stores only posYM (the placement's own vertical offset); level elevation is intentionally excluded. Adding elevation would float overlays and tote markers metres above the equipment meshes on multi-level warehouses (bug HU-004).

• The 3D scene (HardwareTwin3D) is code-split into its own Vite/three.js chunk and loaded lazily; it does not affect the initial bundle size of other screens.

• The twin reuses the editor's presentational components (EquipmentMesh, FunctionPointMarker) from AutomationTopology3D in read-only mode (no drag gizmos, no waypoint handles, all edit handlers are no-ops). Equipment classification (conveyor vs rack vs sorter) is driven by a one-time load of the master-data equipment library (listEquipment) per warehouse; if that load fails the scene still renders with coarser classification.

• Conveyor jam derivation (ui/src/hardwaretwin/conveyorState.ts): pure module (no React, no three.js) that derives the raw jam signal from the totes already held in the twin — no extra backend calls. Three independent signals are OR-ed: stall (scan timeline not advancing), density (totes per metre of belt path), and HOLD decision. A shared jamUntilRef map in HardwareTwin3D applies the 4 s hysteresis so the React effect (re-arms on each poll) and the per-frame lerp (reads the map) stay decoupled.

• ASRS IN/OUT stub height: the 1 m port stub bodies (INDUCT/DISCHARGE) attached to an ASRS rack and their function-point markers both render at the scene's conveyor height — computed at render time as the maximum heightM across all conveyor-family equipment in the scene — rather than at the rack's own heightM (≈ 10 m). This keeps the stubs and their cone/diamond glyphs flush with the conveyor system they feed in both the editor and the twin.

• Conveyor node positions (loadConveyorNodePositions) are loaded once alongside the topology; if the endpoint is absent or returns no projected nodes, scan-replay degrades gracefully — totes fall back to strict equipment anchors rather than crashing.

Operator in-app help

The screen has a Help button (top bar) that opens a contextual drawer with a task-first guide for floor operators: what to watch for, worked examples using realistic demo identifiers, and if/then troubleshooting for the common stuck-tote and equipment-fault scenarios.

Related

Transport Overview · Equipment Integration · Goods-to-Person Stations · Services