Problems with running the pretrained lerobot/smolvla_base

[brickerino]

System Info

- lerobot version: 0.3.4
- Platform: macOS-15.6.1-arm64-arm-64bit
- Python version: 3.10.19
- Huggingface Hub version: 0.35.3
- Datasets version: 4.1.1
- Numpy version: 2.2.6
- PyTorch version: 2.7.1
- Is PyTorch built with CUDA support?: False
- Cuda version: N/A
- GPU model: N/A
- Using GPU in script?: mps

Information

• One of the scripts in the examples/ folder of LeRobot
• My own task or dataset (give details below)

Reproduction

I'm trying to run a pretrained SmolVLA policy on SO-101. I'm launching the following scripts:

python -m lerobot.async_inference.policy_server
--host=127.0.0.1
--port=8080

python -m lerobot.async_inference.robot_client
--server_address=127.0.0.1:8080
--robot.type=so101_follower
--robot.port=/dev/tty.usbmodem5A7C1233791
--robot.id=follower_arm
--robot.cameras="{ camera2 : {type: opencv, index_or_path: 0, width: 1920, height: 1080, fps: 30},
camera1 : {type: opencv, index_or_path: 1, width: 1920, height: 1080, fps: 30}}"
--task="put the toy in the box"
--policy_type=smolvla
--pretrained_name_or_path=lerobot/smolvla_base
--policy_device=mps
--actions_per_chunk=50
--chunk_size_threshold=0.5
--aggregate_fn_name=weighted_average
--debug_visualize_queue_size=True

After launching the scripts the robot freezes and kinda barely fluctuates around the neutral position.
I looked at the actions and they are all in range [-5, 5], while as far as I understand the robot expects actions in range [-100, 100], at least this is the range that the teleopereation script operates with (btw the teleoperation script works properly). I tried scaling these actions up by up to x10, but the robot just started flailing around.

So my theories so far:

1. There is a problem with scale/normalization/representation of actions or observations due to a change in the codebase.
2. I didn't forward the cameras properly (the model requires 3 cameras, but most of the datasets are only on 2, the names of the cameras in the script are different from the documentation, etc). I did try changing indices, adding third cameras though, but didn't work out.

So if you know the problem please let me know before I went into deep debugging.

Expected behavior

One of the following:

1. Robot executes the task.
2. Robot fails but interestingly, and by interestingly I mean it does something, tries to grab, moves to one side and gets stuck there, something.

[brickerino]

Ok so the problem happens because during the inference robot actions are called 'action', and in the post-processing pipeline there is a step for 'so100-blue.buffer.action', 'so100-red.buffer.action' and 'so100.buffer.action' actions. If I put any of these stats under the name 'action' the robot starts to fail properly. What stats am I supposed to use? Is there any meaningful difference?

[brickerino]

And there is normalization for state in the preprocessor config, but there are no stats for that. I guess I could use stats from actions as they are in the same space and technically should follow each other, but the robot still doesn't appear to be working.

[DarioRepoRuler]

Are there any updates on this?