Async Inference Runtime Error when running pi0.5 policy server on GPU device and robot_client on CPU device

[Llkhhb]

I tried to run policy server on GPU device, and robot client on CPU device, I got error below:

INFO 2025-10-08 08:47:20 a_opencv.py:179 OpenCVCamera(1) connected.
INFO 2025-10-08 08:47:22 a_opencv.py:179 OpenCVCamera(2) connected.
INFO 2025-10-08 08:47:22 follower.py:104 follower_arm SO101Follower connected.
WARNING 2025-10-08 08:47:22 ils\utils.py:47 No accelerated backend detected. Using default cpu, this will be slow.
WARNING 2025-10-08 08:47:22 \policies.py:81 Device 'cuda' is not available. Switching to 'cpu'.
WARNING 2025-10-08 08:47:22 ils\utils.py:47 No accelerated backend detected. Using default cpu, this will be slow.
WARNING 2025-10-08 08:47:22 \policies.py:81 Device 'cuda' is not available. Switching to 'cpu'.
INFO 2025-10-08 08:47:22 t_client.py:123 Initializing client to connect to server at
INFO 2025-10-08 08:47:22 t_client.py:142 Robot connected and ready
INFO 2025-10-08 08:47:23 t_client.py:165 Sending policy instructions to policy server
INFO 2025-10-08 08:47:52 t_client.py:488 Starting action receiver thread...
INFO 2025-10-08 08:47:52 t_client.py:456 Control loop thread starting
INFO 2025-10-08 08:47:52 t_client.py:282 Action receiving thread starting
Exception in thread Thread-1 (receive_actions):
Traceback (most recent call last):
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.2800.0_x64__qbz5n2kfra8p0\Lib\threading.py", line 1075, in _bootstrap_inner
self.run()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.2800.0_x64__qbz5n2kfra8p0\Lib\threading.py", line 1012, in run
self._target(*self._args, **self._kwargs)
File "C:\Projects\lerobot_pi05\lerobot\src\lerobot\async_inference\robot_client.py", line 295, in receive_actions
timed_actions = pickle.loads(actions_chunk.data) # nosec
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Projects\lerobot_pi05\lerobot_env05\Lib\site-packages\torch\storage.py", line 530, in _load_from_bytes
return torch.load(io.BytesIO(b), weights_only=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Projects\lerobot_pi05\lerobot_env05\Lib\site-packages\torch\serialization.py", line 1549, in load
return _legacy_load(
^^^^^^^^^^^^^
File "C:\Projects\lerobot_pi05\lerobot_env05\Lib\site-packages\torch\serialization.py", line 1807, in _legacy_load
result = unpickler.load()
^^^^^^^^^^^^^^^^
File "C:\Projects\lerobot_pi05\lerobot_env05\Lib\site-packages\torch\serialization.py", line 1742, in persistent_load
obj = restore_location(obj, location)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Projects\lerobot_pi05\lerobot_env05\Lib\site-packages\torch\serialization.py", line 698, in default_restore_location
result = fn(storage, location)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Projects\lerobot_pi05\lerobot_env05\Lib\site-packages\torch\serialization.py", line 636, in _deserialize
device = _validate_device(location, backend_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Projects\lerobot_pi05\lerobot_env05\Lib\site-packages\torch\serialization.py", line 605, in _validate_device
raise RuntimeError(
RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.

[pkooij]

Hi, we yesterday pushed a fix for async inference: #2132. Your error message however is saying you don't have coda on your machine. Can I ask which platform you are using?

[Llkhhb]

@pkooij hi,

I am using the updated codes merged few hours ago. I tried to run the async inference :
policy_server on linux GPU Server with Cuda and robot_client on CPU Window 11 Laptop without GPU/CUDA
but getting the error :
File "C:\Projects\lerobot_pi05\lerobot_env05\Lib\site-packages\torch\serialization.py", line 605, in _validate_device
raise RuntimeError(
RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.

Many thanks for your reply.

[Llkhhb]

@pkooij I tried to fix error by modifying the policy_server:

for action in action_chunk:
if hasattr(action, 'action') and hasattr(action.action, 'cpu'):
action.action = action.action.cpu()

and now the pi0.5 inference works and generates action for me (I know this code modification may not correct/optimized so I appreciate if you have any suggestion)

[fracapuano]

ciao @Llkhhb 👋
any chance you can attach the policy_server config you're using? From the logger:

WARNING 2025-10-08 08:47:22 \policies.py:81 Device 'cuda' is not available. Switching to 'cpu'.

It seems you might be specifying you wish to run your model on cuda although the cuda backend is not available on your own machine. This won't work unfortunately :( If you have access to a Silicon Mac you can try to run the same policy with a "mps" backend for some acceleration, but you will have the best possible speedup exclusively with "cuda" I am afraid

[fracapuano]

@pkooij I tried to fix error by modifying the policy_server:

for action in action_chunk: if hasattr(action, 'action') and hasattr(action.action, 'cpu'): action.action = action.action.cpu()

and now the pi0.5 inference works and generates action for me (I know this code modification may not correct/optimized so I appreciate if you have any suggestion)

This is also a bit exotic---what is the rationale behind action.action.cpu? all you're doing is running everything on CPU, which you can do without needing to modify the policy server, by providing "device": "cpu" as an argument to the robot client (which defines all operational details for the policy server)

[Llkhhb]

@fracapuano I am trying to convert the action chunk into the format which is compatible with CPU (for non CUDA device) to bypass the RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU

Original files which getting error:

policy_server.py

robot_client.py

My modified file which solved runtime error:

policy_server_modified.py

[Llkhhb]

@pkooij I tried to fix error by modifying the policy_server:
for action in action_chunk: if hasattr(action, 'action') and hasattr(action.action, 'cpu'): action.action = action.action.cpu()
and now the pi0.5 inference works and generates action for me (I know this code modification may not correct/optimized so I appreciate if you have any suggestion)

This is also a bit exotic---what is the rationale behind action.action.cpu? all you're doing is running everything on CPU, which you can do without needing to modify the policy server, by providing "device": "cpu" as an argument to the robot client (which defines all operational details for the policy server)

I am not sure whether I do it correctly or not,
in the policy server side, I am trying to move the action chunk from GPU to CPU, before sending the action chunk to non-CUDA client

[Llkhhb]

ciao @Llkhhb 👋 any chance you can attach the policy_server config you're using? From the logger:

WARNING 2025-10-08 08:47:22 \policies.py:81 Device 'cuda' is not available. Switching to 'cpu'.

It seems you might be specifying you wish to run your model on cuda although the cuda backend is not available on your own machine. This won't work unfortunately :( If you have access to a Silicon Mac you can try to run the same policy with a "mps" backend for some acceleration, but you will have the best possible speedup exclusively with "cuda" I am afraid

@fracapuano

WARNING 2025-10-08 08:47:22 \policies.py:81 Device 'cuda' is not available. Switching to 'cpu'.

the warning message occurs because my robot_client is CPU device, so I offload the heavy computations to policy server with GPU device, but I confirmed that the policy_server is running on GPU with CUDA correctly, I can see the GPU memory usage (GBs) when running the inference using nvidia-smi:

INFO 2025-10-09 10:24:24 y_server.py:138 Receiving policy instructions from ipv4:xxxxxx | Policy type: pi05 | Pretrained name or path: models/pi05 | Actions per chunk: 200 | Device: cuda

[fracapuano]

Great! So everything is working as expected I reckon @Llkhhb 🥳 Still, any chance you can share the configuration you use for both the policy server and robot client?

[fracapuano]

Closing this issue as this was addressed! Feel free to reopen if necessary :)

[kevinjosethomas]

I think the issue here is when our policy server is a CUDA machine but the client server is a non-CUDA machine.

I'm running into the same error when I have my policy running on a CUDA machine but I'm using my Macbook to connect to my SO101s:

python3 -m lerobot.async_inference.robot_client \
    --server_address=0.0.0.0:8080 \
    --robot.type=so101_follower \
    --robot.port=/dev/tty.usbmodem5A7C1169541 \
    --robot.id=arm \
    --robot.cameras="{ top: {type: opencv, index_or_path: 2, width: 1920, height: 1080, fps: 30}, wrist: {type: opencv, index_or_path: 0, width: 1920, height: 1080, fps: 30}}" \
    --task="Pick up the orange and place it on the plate" \
    --policy_type=smolvla \
    --pretrained_name_or_path=kevinjosethomas/smolvla_orange_plate \
    --policy_device=cuda \
    --actions_per_chunk=50 \
    --chunk_size_threshold=0.5 \
    --aggregate_fn_name=weighted_average \
    --debug_visualize_queue_size=True
INFO 2025-10-31 10:45:01 t_client.py:470 {'actions_per_chunk': 50,
 'aggregate_fn_name': 'weighted_average',
 'chunk_size_threshold': 0.5,
 'debug_visualize_queue_size': True,
 'fps': 30,
 'policy_device': 'cuda',
 'policy_type': 'smolvla',
 'pretrained_name_or_path': 'kevinjosethomas/smolvla_orange_plate',
 'robot': {'calibration_dir': None,
           'cameras': {'top': {'color_mode': <ColorMode.RGB: 'rgb'>,
                               'fourcc': None,
                               'fps': 30,
                               'height': 1080,
                               'index_or_path': 2,
                               'rotation': <Cv2Rotation.NO_ROTATION: 0>,
                               'warmup_s': 1,
                               'width': 1920},
                       'wrist': {'color_mode': <ColorMode.RGB: 'rgb'>,
                                 'fourcc': None,
                                 'fps': 30,
                                 'height': 1080,
                                 'index_or_path': 0,
                                 'rotation': <Cv2Rotation.NO_ROTATION: 0>,
                                 'warmup_s': 1,
                                 'width': 1920}},
           'disable_torque_on_disconnect': True,
           'id': 'arm',
           'max_relative_target': None,
           'port': '/dev/tty.usbmodem5A7C1169541',
           'use_degrees': False},
 'server_address': '0.0.0.0:8080',
 'task': 'Pick up the orange and place it on the plate'}
INFO 2025-10-31 10:45:05 a_opencv.py:180 OpenCVCamera(2) connected.
INFO 2025-10-31 10:45:07 a_opencv.py:180 OpenCVCamera(0) connected.
INFO 2025-10-31 10:45:07 follower.py:104 arm SO101Follower connected.
INFO 2025-10-31 10:45:07 t_client.py:113 Initializing client to connect to server at 0.0.0.0:8080
INFO 2025-10-31 10:45:07 t_client.py:132 Robot connected and ready
INFO 2025-10-31 10:45:07 t_client.py:155 Sending policy instructions to policy server
INFO 2025-10-31 10:45:26 t_client.py:478 Starting action receiver thread...
INFO 2025-10-31 10:45:26 t_client.py:446 Control loop thread starting
INFO 2025-10-31 10:45:26 t_client.py:272 Action receiving thread starting
Exception in thread Thread-1 (receive_actions):
Traceback (most recent call last):
  File "/opt/anaconda3/envs/lerobot/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/opt/anaconda3/envs/lerobot/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/kevinjosethomas/Documents/Code/Major/lerobot/src/lerobot/async_inference/robot_client.py", line 285, in receive_actions
    timed_actions = pickle.loads(actions_chunk.data)  # nosec
  File "/opt/anaconda3/envs/lerobot/lib/python3.10/site-packages/torch/storage.py", line 530, in _load_from_bytes
    return torch.load(io.BytesIO(b), weights_only=False)
  File "/opt/anaconda3/envs/lerobot/lib/python3.10/site-packages/torch/serialization.py", line 1549, in load
    return _legacy_load(
  File "/opt/anaconda3/envs/lerobot/lib/python3.10/site-packages/torch/serialization.py", line 1807, in _legacy_load
    result = unpickler.load()
  File "/opt/anaconda3/envs/lerobot/lib/python3.10/site-packages/torch/serialization.py", line 1742, in persistent_load
    obj = restore_location(obj, location)
  File "/opt/anaconda3/envs/lerobot/lib/python3.10/site-packages/torch/serialization.py", line 698, in default_restore_location
    result = fn(storage, location)
  File "/opt/anaconda3/envs/lerobot/lib/python3.10/site-packages/torch/serialization.py", line 636, in _deserialize
    device = _validate_device(location, backend_name)
  File "/opt/anaconda3/envs/lerobot/lib/python3.10/site-packages/torch/serialization.py", line 605, in _validate_device
    raise RuntimeError(
RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.