Skip to content

Review Cortex2

Jump to bottom
hwoo.han edited this page Aug 11, 2026 · 1 revision

In-Depth Review β€” Cortex 2.0: Planning Before Acting (Sereact)

Model: Cortex 2.0 β€” a VLA augmented with a world-model foresight-planning layer (modular WAM+VLA hybrid) Β· Sereact GmbH (Stuttgart) Paper: "Cortex 2.0: Grounding World Models in Real-World Industrial Deployment" β€” arXiv 2604.20246 (Apr 22 2026) Β· blog Status: arXiv preprint β€” results are company/deployment-reported from Sereact's production fleet, not independently replicated. Filed under Latest Papers. Backing: $110M Series B (2026), scaling Cortex 2 + US expansion.

The blog's headline: "Today's Cortex sees and picks; Cortex 2.0 thinks first, then acts." It is the modular/bolted-on exemplar of the WAM+VLA hybrid (NVIDIA WAM thesis Β§2.6) β€” a world model added as a foresight-planning layer in front of a VLA action head. Companion: VLA Architectures Β§4.2b Β· World Models Β· MOTUS Β· Being-H0.7.

1. TL;DR

  1. A world model as a planning layer bolted onto a VLA. Four hierarchical stages: (1) High-level VLM (a 2B-VLM) encodes the scene into task context β†’ (2) World Model generates k candidate future trajectories in visual latent space via flow matching β†’ (3) PRO (Process-Reward Operator) scores each β†’ (4) flow-matching action head commits to the best branch and replans in real time at 30 Hz.
  2. Scoring by progress, risk, termination. PRO uses frozen heads (trained on deployment data): Progress (Ξ” value V_Ο†), Risk (failure probability, penalizing high-speed contact / compression / edge impacts), Termination (success likelihood). Composite S_j = Ξ”p βˆ’ λ·ρ + Ξ²Β·d; pick argmax, then feed a binarized advantage signal into the policy.
  3. Adaptive foresight = a compute dial. More rollouts when failure is expensive (packing, fragile placement), fewer when recovery is cheap (regrasp). Success rises 0.962 (k=1) β†’ 0.996 (k=30) while step latency grows 310 ms β†’ 9,200 ms.
  4. Strong deployment-reported wins over Ο€0.5 / Diffusion Policy / RDT-2 across four industrial tasks β€” with 0 human interventions where baselines needed dozens.

2. Why it matters

  • It makes "world-model foresight" a deployable planning layer, not a research toy. Cortex 2.0 keeps a fast VLA action head and only scores imagined branches β€” a pragmatic answer to the WAM latency problem (Review-WAM-vs-VLA-Robustness): imagination is used for selection, and the compute is a tunable dial per task.
  • It is the clearest industrial datapoint for the modular hybrid. Contrast the unified hybrids (MOTUS, Being-H0.7, Cosmos 3): Cortex 2.0 bolts the world model on as a process-reward planner, closer in spirit to test-time search (VLA-Reasoner) than to a single MoT.
  • Trained on real deployment at scale (>10M episodes / >25k h of warehouse operation) β€” the data regime academic hybrids can't access.

3. Architecture

flowchart LR
  O[observation] --> VLM[High-level VLM Β· 2B<br/>task context s_t]
  VLM --> WM[World model<br/>k candidate futures in visual latent<br/>flow matching, ODE Οƒ:0β†’1]
  WM --> PRO[PRO scorer<br/>progress βˆ’ λ·risk + Ξ²Β·termination]
  PRO -->|argmax β†’ binarized advantage I_t| AH[Flow-matching action head]
  AH ==>|30 Hz, real-time replan| ACT[action]
Loading
  • Candidate generation. Conditioned on current latent z_t and task context s_t, each of k rollouts starts from a distinct noise draw ΞΎβ½Κ²βΎβˆΌπ’©(0,I), integrated by ODE from Οƒ=0β†’1 over horizon H_wm.
  • PRO (Process-Reward Operator). Three frozen heads score each rollout: progress Ξ”p, risk ρ, termination d β†’ S_j = Ξ”p βˆ’ λρ + Ξ²d. The winning branch's advantage is binarized I_t∈{0,1} and fed to the policy as conditioning.
  • Embodiment adaptation is handled entirely by the action heads (learned projections W_z, W_I), so the planner transfers across platforms.
  • Data: deployment >10M episodes / >25k h (warehouse), teleop ~40k/~400 h, open-source ~970k/~2k h (OXE, BridgeData V2, DROID), synthetic RoboCasa ~20k. Baselines trained at equal 200 GPU-hour budgets.

4. Results (deployment-reported; success / completion time / human interventions)

Task (dual-arm unless noted) Cortex 2.0 Ο€0.5 Diffusion Policy RDT-2
Single-arm pick-and-place (16 trials) 0.98 Β· 20 s Β· 0 0.7 Β· 49 s Β· 2 0.56 Β· 53 s Β· 4 0.4 Β· 63 s Β· 7
Sorting items & trash (8.7k ep) 0.95 Β· 700 s Β· 0 0.61 Β· DNF Β· 53 0.47 Β· DNF Β· 59 0.18 Β· DNF Β· 95
Sorting screws (3.1k ep) 0.98 Β· 180 s Β· 0 0.4 Β· DNF Β· 24 0.2 Β· DNF Β· 16 0.0 Β· DNF Β· 50
Shoebox unpacking (2.9k ep) 0.96 Β· 58 s Β· 0 0.6 Β· 103 s Β· 5 0.12 Β· 52 s Β· 9 0.0 Β· 62 s Β· 10

(DNF = did not complete; per-operation success for the multi-item tasks.)

5. Significance & limitations

Significance. Cortex 2.0 shows world-model foresight-as-planning working on a real industrial fleet with zero interventions β€” the strongest deployment evidence yet for the modular WAM+VLA hybrid, and a clean separation of imagination (selection) from control (fast action head).

Limitations.

  1. Company/deployment-reported, not peer-reviewed or independently replicated. Results come from Sereact's own production stack and tasks.
  2. Proprietary data & environments. The >10M-episode deployment corpus is closed; generalization outside Sereact deployments is untested.
  3. Latency–foresight tradeoff is steep. k=30 reaches 0.996 but at 9.2 s/step β€” high-k foresight is viable only where failure cost dominates cycle time.
  4. Per-task tuning. The advantage-binarization threshold Ο΅(s_t) and the k/H_wm budget are hand-set per task.
  5. No standardized benchmark. Baselines are re-trained in-house at matched compute, but there's no shared LIBERO/RoboTwin comparison.

6. Links

← Back to Latest Papers Β· Home Β· Reviews

Navigation

πŸ“– Reviews

🏷 Model lineages

🧠 ML foundations

πŸ—“ Conferences

(each page indexes its per-paper pages)

πŸ“Œ Foundational

Clone this wiki locally