Hardware Visualisation

Hardware Visualisation (Digital Twin)

The Hardware Visualisation screen (/hardware-twin) renders the saved automation topology read-only in 3D and overlays a live picture of equipment activity and handling-unit movement derived from the device-task feed — giving a real-time digital twin of the physical system without any extra backend instrumentation.

Backed by GET /api/flow/device-tasks (flow-orchestrator, port 8085), polled every 2 s. RBAC: ADMIN, SUPERVISOR, or OPERATOR role required.

Equipment state

The base scene is rendered using the same meshes as the Automation Topology Editor (two-tone conveyor ribbons, racked ASRS, sorter/workstation boxes, SCAN/QRY/DIV function-point markers and labels). Runtime state is shown as a clean overlay on top of the editor scene. The overlay type differs by equipment category.

Conveyor belt skins

Conveyors wear their state as a belt skin — a thin emissive layer riding just above the belt surface (hardwaretwin/conveyorState.ts, HardwareTwin3D.tsx: ConveyorStateSkin). State changes ease via a brief colour lerp (τ ≈ 0.35 s) instead of snapping:

Belt skin colour	State	Trigger
Green	Functional	No fault; normal (or no) traffic on this segment. A subtle tint so a healthy floor stays readable.
Orange	Jam / heavy traffic	Any of: (a) a stalled tote — its scan timeline has not advanced for > 2.5× the expected hop time; (b) tote density ≥ 0.5 totes/m of belt path length (min 2 totes); or (c) a HELD divert decision on a tote currently riding this belt. A 4 s hysteresis holds the orange state after the raw signal clears so borderline polls do not strobe the skin.
Red	Fault	An active device task on this equipment reported FAILED. Red takes priority over orange over green.

Attribution — which belt a tote "rides" — uses buildBeltLocator from motion.ts: the same nearest-belt projection the path resolver uses, so attribution and tote motion always agree. Skin geometry uses the editor's effectiveSections (the same directed-segment breakdown as the Reporting traffic heatmap). Skins are raycast-transparent so clicking a belt still selects the conveyor body underneath.

Non-conveyor equipment overlays

Non-conveyor equipment (ASRS racks, workstations, sorters without a skinnable belt body) retains the floating activity sphere overlay:

State	Overlay	Trigger
Idle	None — base mesh only	No active task seen in the current window
Running	Floating amber sphere above the equipment, pulsing in size and brightness	At least one active (REQUESTED/DISPATCHED) task on this equipment
Faulted	Floating red sphere above the equipment (steady)	Any FAILED task on this equipment in the current window

Selected equipment of any type shows a lime floor ring below it.

Tote (handling-unit) markers

The twin shows two distinct layers of handling-unit markers that together give a complete inventory picture of the warehouse at a glance.

Live totes (in-transit, queued, done)

Active handling units are drawn as 3D tote models (600 × 400 × 320 mm open-top container — base plate, four slightly tapered walls, and a bright state-coloured rim) whose positions replay the observed scan trail — no motion is invented.

In-transit positioning (ADR-0008 3d-3, smooth motion): when a CONVEY task is running, the twin fetches the hu_transport_trace rows for that HU each poll tick and feeds every SCANNED row (node, routed-to next node, timestamp) into a per-tote interpolation buffer (ui/src/hardwaretwin/motion.ts, pure functions). The 3D layer renders each tote at now − 3.5 s (just over one poll interval) and interpolates only between two known buffered waypoints, following the conveyor's polyline geometry rather than the straight chord — so polling cadence never shows as freeze-then-jump, and per-hop speed reproduces exactly what the floor reported. If the buffer runs dry (no fresh poll yet), the tote dead-reckons gently toward the already-known next node at the last observed speed and eases to a stop; when revised data arrives, the position error is blended away over ~0.5 s instead of snapping. Genuine discontinuities (rack store/retrieve, induction; hops > 6 m or trace gaps > 20 s) still teleport — that is real. If no scan trace is available, the tote falls back to its strict equipment anchor (explicit equipmentId on the task only; no family guess). A tote with neither an observed scan nor a direct equipment anchor is hidden rather than guessed.

Queued totes: the queue state is sourced from the real induction queue per GTP workstation (flow's getStationQueue API), so a tote that has finished all work simply disappears rather than lingering forever as "queued" from a stale completed CONVEY task. The twin falls back to task-inference when queue data is unavailable.

Equipment-activity anchoring still uses a family-level fallback (ASRS/AUTOSTORE/AMR → storage placement; CONVEYOR → longest-path conveyor) but only for the amber/red activity sphere overlay, never for tote-marker positions.

Totes at non-conveyor equipment (workstations, ASRS placements) sit on the top surface of that equipment's bounding box so they are visibly above the solid mesh.

Tote state	Colour	Meaning
In-transit	Bright state-coloured rim	Tote has an active CONVEY task; position follows the scanned node trail
Recirculating	Bright state-coloured rim	Last sorter decision was RECIRCULATED or HELD; loops back to induct
Queued	Bright state-coloured rim	Tote is in the real induction queue at a GTP workstation
Done	Bright state-coloured rim	Tasks completed; marker lingers briefly then is removed

Stored totes — rack contents (ADR-0009 §5)

HUs that are at rest in ASRS storage are shown as bright ice-blue mini-totes rendered directly inside the rack mesh at their exact cell position. This is a pure representation of the live HU registry (populated via transport lifecycle — see Inbound and Inventory); no position is invented or guessed:

• Cell X spreads the tote along the rack's length.
• Shuttle level (posY) stacks totes up the rack height.
• Side (L/R) places the tote in the correct rack flank relative to the aisle.
• Channel depth (posZ) pushes the tote outward from the aisle face.

HUs whose master-data location lacks cell coordinates (posX/posY) are skipped entirely. HUs that already appear as live totes (above) are excluded to prevent duplicates.

Clicking a stored-tote mini opens the same HU detail / trace panel as clicking a live tote. When selected, a small label showing the HU code floats above the mini.

Master-data locations (cell coordinates) are loaded once per warehouse alongside the topology; the HU registry is re-fetched on every poll tick and deduplicated against the live-tote set.

Stats bar

A bar across the top of the screen shows live aggregate counters derived from the current device-task window:

Counter	Description
In-transit	Totes with at least one active transport task
Queued	Totes queued at a workstation
Throughput/min	Completed tasks in the last 60 s
Recirculations	Sorter RECIRCULATED decisions in the current window
Faults	Equipment with at least one FAILED task
In storage	Totes currently at rest in ASRS rack cells (live HU registry)

Level filter

When the topology spans multiple floor levels, a level selector in the header restricts the scene to a single level. All unrelated equipment and tote markers are hidden; the stats bar counts only the visible level.

Labels toggle

A Labels checkbox in the control bar (default off) shows or hides the text labels that float above each equipment placement and function-point marker:

Setting	Effect
Off (default)	All decorative overlays are hidden: equipment-code text, SCAN/QRY/DIV function-point cones/diamonds, and conveyor junction rings. Only the base 3D meshes, tote markers, and activity/state overlays remain. Maximum signal-to-noise at a glance.
On	Equipment codes appear above each placement; SCAN/QRY/DIV labels and cone/diamond glyphs appear above each function-point; conveyor junction rings are shown. Matches the Automation Topology Editor view.

The toggle does not affect the 3D geometry, tote markers, or activity/state overlay indicators.

Interactions

Action	Result
Click equipment	Opens a detail panel with the equipment's recent device tasks
Click tote marker	Opens the HU trace panel for that handling unit (same trace as Transport Overview#per-hu-transport-trace)
Click background / grid	Clears both selections
Left-drag	Orbit the camera
Right-drag	Pan
Scroll / middle-drag	Zoom

Implementation notes

• Data sources per poll tick:

  Source	Endpoint	Purpose
  Device tasks	GET /api/flow/device-tasks	Equipment activity + tote state
  HU transport trace	GET /api/flow/hu-trace?huId= (fan-out, ≤ 12 active HUs)	SCANNED rows → in-transit tote positions
  Station queue	GET /api/flow/gtp/queue?stationId= (per workstation)	Real queued-tote truth source
  Handling units	GET /api/inventory/handling-units	Full HU registry → stored-tote positions inside the rack

• Data sources loaded once per warehouse (on topology load):

  Source	Endpoint	Purpose
  Conveyor topology	GET /api/flow/conveyor/topology?warehouseId=	Routing-node world positions for scan-replay
  Master-data locations	GET /api/masterdata/locations?warehouseId=	Cell coordinates (posX, posY, posZ, side) used by deriveStoredTotes

• Coordinate system: all twin geometry is floor-relative — matching the Automation Topology Editor, which renders each level on the floor plane regardless of the level's elevationM. The PlacementGeom.baseY field stores only posYM (the placement's own vertical offset); level elevation is intentionally excluded. Adding elevation would float overlays and tote markers metres above the equipment meshes on multi-level warehouses (bug HU-004).

• The 3D scene (HardwareTwin3D) is code-split into its own Vite/three.js chunk and loaded lazily; it does not affect the initial bundle size of other screens.

• The twin reuses the editor's presentational components (EquipmentMesh, FunctionPointMarker) from AutomationTopology3D in read-only mode (no drag gizmos, no waypoint handles, all edit handlers are no-ops). Equipment classification (conveyor vs rack vs sorter) is driven by a one-time load of the master-data equipment library (listEquipment) per warehouse; if that load fails the scene still renders with coarser classification.

• Conveyor jam derivation (ui/src/hardwaretwin/conveyorState.ts): pure module (no React, no three.js) that derives the raw jam signal from the totes already held in the twin — no extra backend calls. Three independent signals are OR-ed: stall (scan timeline not advancing), density (totes per metre of belt path), and HOLD decision. A shared jamUntilRef map in HardwareTwin3D applies the 4 s hysteresis so the React effect (re-arms on each poll) and the per-frame lerp (reads the map) stay decoupled.

• ASRS IN/OUT stub height: the 1 m port stub bodies (INDUCT/DISCHARGE) attached to an ASRS rack and their function-point markers both render at the scene's conveyor height — computed at render time as the maximum heightM across all conveyor-family equipment in the scene — rather than at the rack's own heightM (≈ 10 m). This keeps the stubs and their cone/diamond glyphs flush with the conveyor system they feed in both the editor and the twin.

• Conveyor node positions (loadConveyorNodePositions) are loaded once alongside the topology; if the endpoint is absent or returns no projected nodes, scan-replay degrades gracefully — totes fall back to strict equipment anchors rather than crashing.

Operator in-app help

The screen has a Help button (top bar) that opens a contextual drawer with a task-first guide for floor operators: what to watch for, worked examples using realistic demo identifiers, and if/then troubleshooting for the common stuck-tote and equipment-fault scenarios.

Related

Transport Overview · Equipment Integration · Goods-to-Person Stations · Services