Skip to content

RACS V1 — Seeded Robustness Evaluation

Latest

Choose a tag to compare

@ridwanbale ridwanbale released this 07 Sep 22:48

RACS V1 is the first reproducible experimental release of the Risk-Aware Coordination System (RACS), an open coordination layer for autonomous operations.

What V1 demonstrates

V1 evaluates whether observable operational degradation in an AMR-based warehouse can be detected early enough for a separate risk-aware coordination layer to intervene before or during downstream propagation.

The experiment includes:

  • deterministic warehouse task/service simulation
  • progressive AMR service degradation
  • conventional reactive hard-failure recovery baseline
  • observable robot-level anomaly attribution
  • predictive RACS drain/cordon coordination
  • downstream workstation propagation modeling
  • paired healthy counterfactuals for downstream attribution
  • disturbance-time WIP resilience conditions
  • seeded workload-timing variation
  • reproducible experiment outputs and plotting utilities

Robustness evaluation

The frozen V1 study uses:

  • 30 paired workload seeds (1000–1029)
  • 4 disturbance-time WIP conditions (0, 1, 2, 4)
  • healthy, reactive-baseline, and RACS conditions
  • 360 total simulation runs
  • identical workload realization within every paired comparison

Main results

H1 — Transport coordination: partially supported

RACS improved mean queue AUC and mean task latency across the seeded study, although individual workload realizations included regressions.

  • Mean queue AUC delta, RACS − baseline: -2.9333
  • Mean latency delta, RACS − baseline: -0.0901
  • Queue AUC worsened in 24 / 120 paired comparisons

H2 — Greater downstream benefit when intervention precedes propagation: not established

Downstream benefit was not greater in the intervention-before-propagation group than in the at/after-propagation group.

H3 — Downstream WIP and intervention headroom: supported within tested conditions

Within the tested WIP levels, greater disturbance-time WIP was associated with a higher fraction of runs in which RACS intervened before downstream propagation:

  • WIP 0: 66.7%
  • WIP 1: 80.0%
  • WIP 2: 93.3%
  • WIP 4: 93.3%

One cascade-prevention case was observed at WIP 4. This is reported as a single observed outcome, not as a general claim that RACS prevents cascading failures.

Reproducibility

The experiment implementation was frozen before public result packaging.

Frozen implementation snapshot:

b4764da6193e013c3c2e1f86763597498bc6253f

V1 release tip:

0fc41a0

The repository includes:

  • full experiment methodology
  • compact reference-result artifacts
  • generated figures
  • plotting utility
  • deterministic seeded workload generation
  • automated tests

Limitations

V1 is a discrete-event systems experiment, not a physical-robot validation.

Current limitations include:

  • single-site warehouse simulation
  • one degrading robot
  • service-capacity degradation rather than locomotion dynamics
  • deterministic robot service duration
  • abstract downstream workstation model
  • only workload arrival timing randomized
  • 30 seeds with descriptive statistics only
  • no statistical-significance claim
  • no ROS 2 / Gazebo / physical robot validation
  • downstream effect sizes remain small

The V1 results are intended as a reproducible baseline for further independent evaluation and extension.