how to control a so-101 with trained ACT model?

[initia1013]

https://huggingface.co/initie/test_pick_result
This is my pre-trained model for grabbing the switch on the desk by ACT model.
How to run this policy model on the Anaconda?
Already by way of example,

python -m lerobot.record --robot.type=so101_follower
--robot.port=COM3
--robot.id=ammd_follower_arm
--robot.cameras="{ front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}, side: {type: opencv, index_or_path: 1, width: 640, height: 480, fps: 30} }"
--display_data=True
--dataset.repo_id="initie/eval_test_pick"
--dataset.single_task="Grab the switch"
--policy.path=initie/test_pick_result
--teleop.type=so101_leader --teleop.port=COM5
--teleop.id=ammd_leader_arm --dataset.reset_time_s=5

This is the example code from Lerobot tutorial, but when i run these codes, I had to record 10 episodes again.
I just wanna run a pre-trained model, not record an episode again. I'm curious about a simple code that only "runs" that model not including recording

[milong26]

follow tutorial evaluate run command will be like

python -m lerobot.record --robot.type=so101_follower
--robot.port=COM3
--robot.id=ammd_follower_arm
--robot.cameras="{ front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}, side: {type: opencv, index_or_path: 1, width: 640, height: 480, fps: 30} }"
--display_data=True
--dataset.repo_id="initie/eval_test_pick"
--dataset.single_task="Grab the switch"
--policy.path=initie/test_pick_result

Don't set teleop if you want to control it with model

[initia1013]

I used the same code as yours.but it includes recording execution function so I wanna get rid of it and just run the policy

[milong26]

@initia1013 why u want to skip recording? If you really want, maybe you need to write script on your own, follow record_loop,you need to prepare observation.
opservation_frame -> predict_acion ->action_values -> send_action

[initia1013]

I wrote the codes that move the arms in a specific motion, and I wanted to execute the code of that trained model between those motion codes.Thank you for comments

[milong26]

@initia1013 I think you can follow this:

# if policy is given it needs cleaning up
    if policy is not None:
        policy.reset()

    timestamp = 0
    start_episode_t = time.perf_counter()
    while timestamp < control_time_s:
        start_loop_t = time.perf_counter()

        if events["exit_early"]:
            events["exit_early"] = False
            break
        observation = robot.get_observation()
        if policy is not None or dataset is not None:
            observation_frame = build_dataset_frame(dataset.features, observation, prefix="observation")

        if policy is not None:
            action_values = predict_action(
                observation_frame,
                policy,
                get_safe_torch_device(policy.config.device),
                policy.config.use_amp,
                task=single_task,
                robot_type=robot.robot_type,
            )
            action = {key: action_values[i].item() for i, key in enumerate(robot.action_features)}
              
        sent_action = robot.send_action(action)
            
        # fps=30 in DatasetRecordConfig
        dt_s = time.perf_counter() - start_loop_t
        busy_wait(1 / fps - dt_s)

        timestamp = time.perf_counter() - start_episode_t

also start from this action = policy.select_action(observation) in function predict_action is more grounded.

But rewrite is hard :P wish you can finish it

[initia1013]

@milong26 oh thank you friend . I'll make sure to refer to this and complete the whole code. Have a nice day! :)

[AdilZouitine]

hey this issue is solved now? do you need more information? 😄

[initia1013]

no I still haven't solved it:( A chain of errors occurred when I removed the code for the recording and saving functioning codes.. Is there any simple code that is only responsible for only implementing policies?@AdilZouitine

[nccpu66-cpy]

hey, do you solve this issue? I also want to control so101 by using my training smolvla model. Can you give me any advice?Thanks.

[championAI]

hey, do you solve this issue? I also want to control so101 by using my training smolvla model. Can you give me any advice?Thanks.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity (6 months). It will be closed if no further activity occurs. Any change, comment or update to this issue will reset this count. Thank you for your contributions.

[github-actions]

This issue was closed because it has been stalled for 14 days with no activity. Feel free to reopen if is still relevant, or to ping a collaborator if you have any questions.