Adding robocasa server and client for batch inference

[akshay18iitg]

What this does

Integrating robocasa simulator with client and server using web sockets to communicate

How it was tested

Start server with:
python -m opentau.scripts.robocasa_server_async --config_path /path/to/train_config.json --robocasa_action_dim 12 --robocasa_port 8765

Start client with:
MUJOCO_GL=osmesa python -m robocasa.scripts.client_async NavigateKitchen --num-rollouts 30 --num-parallel 5 --host 0.0.0.0 --port 8765

Checklist

• I have added Google-style docstrings to important functions and ensured function parameters are typed.
• My PR includes policy-related changes.
  • If the above is checked: I have run the GPU pytests (pytest -m "gpu") and regression tests.

Note: Before submitting this PR, please read the contributor guideline.

[shuheng-liu]

can we add documentation/tutorial on how to set up Robocasa? I assume it involves setting up some extra dependency

[Copilot]

Pull request overview

Adds a RoboCasa WebSocket-based policy inference integration to OpenTau, enabling remote action inference from RoboCasa observations (including a batched request/response wire protocol) and providing a threaded rollout client.

Changes:

• Introduces an async WebSocket server that loads an OpenTau policy (or runs a stub policy) and serves single and batched inference requests.
• Adds a threaded RoboCasa rollout client that batches observations across worker threads into one WebSocket request per timestep and records rollout results.
• Adds a robocasa scripts package initializer.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.

File	Description
src/opentau/scripts/robocasa/server.py	WebSocket policy server with OpenTau policy loading + batched inference protocol support.
src/opentau/scripts/robocasa/client.py	Threaded rollout client that batches observations/actions over WebSockets and writes rollout summaries/videos.
src/opentau/scripts/robocasa/__init__.py	Declares the robocasa scripts subpackage.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.