Sim-to-real gap with Feetech STS3215

[MarcDcls]

Original question from @AI-FanGe:

Thanks for the great project! I have tried to run your project using another motor the STs3215 C049, but encountered some sim2real problems.
I have tried the training and successfully trained a model, but when I deployed the model, the sim2real gap is pretty huge even though I have used the specific mass of my robot model and the BAM parameters I calculated for the STS3215 motor I used.

The main difference I found is that the speed of the motor deployed in the real world is different from the motor speed in Mujoco. And when the robot performs the walking pattern model, it can not balance on the ground and falls directly.

I would like to ask three questions below, and I really hope that you can help me. I have wasted a couple of weeks in solving the problem.

1. Do you think it is possible to train a model deployed with motor Feetech sts3215 c049 (I mean technically, is it possible with this hardware).
2. How do you control the motor using your model and make sure that the parameters generated by the model is compatiable to your motors? What parameters generated by the model do you use to control the motor?
3. Based on my problems, what kind of solution or any thoughts came to your mind? Please feel free to tell me anything that can help me!

Thanks again!

[MarcDcls]

Hi, thanks for detailing your workflow! Tracking down sim-to-real gaps is usually the hard part, but it is definitely fixable. To answer your questions:

1. Is it possible to use STS3215 motors?

Yes, I think that building a Microban variant with these motors and successfully bridging the sim-to-real gap is technically possible. It is great that you identified your motor parameters using BAM. However, a major source of this gap often comes from motor and sensor delays. Since you switched hardware, you will likely need to remeasure and update these delays for your specific setup.

2. Motor control and BAM parameters

You mentioned a "walking pattern model", so I want to make sure we are on the same page: are you using Reinforcement Learning (RL) for the walking policy?

If you are indeed using RL, you need to train a new ONNX model using the Rhoban/mjlab_microban repository. For this training phase, you will use your updated MJCF file alongside the new BAM actuator model.

Crucially, the BAM parameters are only used in simulation during the training phase. On the real robot, BAM is not used for control at all. The trained policy simply takes the observations and directly outputs position targets for the motors.

3. Next steps

To help you further, I have a few questions: Are you using the RL pipeline for walking? Also, did you perform the BAM identification yourself on your motors, or did you retrieve an existing identification?

Let me know, and we can figure this out!

[AI-FanGe]

Hi Marc, thanks for your detailed reply. Your help was really, really helpful, and I did more experiments recently and encountered more problems; the most important one is the differnence of moter performance between RL model performance in Mujoco and real motor performance in the real world. ( The RL results show that the robot can walk, but it is not working when I deployed the model in a real robot )

Yes, I am using the RL pipeline. I retrained an ONNX policy using my modified MJCF model and the BAM parameters that I identified for my STS3215 C049 motors.

I now understand that BAM is only used during simulation and that the real robot receives position targets.

My main problem is that the real motors respond differently from the simulated motors. Their speed, delay and position tracking behavior are noticeably different. This causes a phase mismatch, so the policy works in simulation but the real robot loses balance and falls almost immediately.

I have a few key questions:

1. How exactly is the ONNX output converted into the final motor position target? Are action scaling, default joint positions, joint directions, offsets, clipping or interpolation applied?

2. Which control settings must match between simulation and hardware, such as control frequency, servo P gain, velocity/acceleration limits, PWM/torque limits and supply voltage?

3. You mentioned motor and sensor delays. How do you measure these delays, and where should they be configured during training?

I also have a more general question about BAM actuator identification. If a motor does not already have a predefined model in BAM, what is the recommended complete procedure for measuring, identifying and validating it?
More specifically:

1. What trajectories, loads, velocities and operating conditions should be used for identification?
2. Which motor and firmware settings must be recorded during the experiment?
3. Does BAM identify only torque and friction parameters, or does it also model the internal position controller, saturation, deadband and back-EMF behavior?
4. Are communication and actuator delays included in BAM, or must they be modeled separately?
5. Should the motor be identified under multiple representative loads and velocities rather than only in an unloaded condition?
6. How do you validate the identified model using trajectories that were not part of the identification dataset?
7. Which metrics do you normally compare between simulation and hardware—position error, velocity error, rise time, phase delay or frequency response—and what level of error is usually acceptable for sim-to-real walking?
   My goal is not to obtain a perfect actuator model under every possible condition. I want to ensure that, within the operating range used by the walking policy, the simulated and real motors produce sufficiently similar position and velocity responses when given the same position-target trajectory.
   Do you have a recommended single-joint or suspended-robot experiment for comparing simulation and hardware before testing the full walking policy? If possible, could you also share an example of the identification and validation workflow used when adding a new motor to BAM?
   I can provide plots and logs containing the target position, measured position, measured velocity and timestamps from both simulation and the real robot if that would help diagnose the problem.
   Thanks again for your help!

[MarcDcls]

Hi again 🙂

Control Pipeline & Sim-to-Real

1. ONNX Output Conversion
The policy outputs position target offsets relative to the default reference position used during RL. In the deployed pipeline, this reference position is stored in the ONNX metadata and is used to compute the final absolute target position sent to the motors. The action scale is 1.0 by default. If you modified it, you must apply that scaling factor on the real robot.

2. Sim-to-Real Checklist
Before digging deeper, check these three points:

• Motor Directions: Ensure the joint directions in simulation are exactly the same as on the real robot. An inverted motor explains direct falls.
• Control Settings & Gains: Every motor parameter used in simulation must match the real hardware. Check your P gains in particular. Ensure the simulated gains match the real ones, taking into account any unit conversion factor between the firmware's internal values and the real-world units (check the motor's datasheet).
• Delays: You can measure sensor delays by running sinusoidal trajectories using the rotate_head movement. There is code in the repository with a specific flag and scripts to plot this data. Once measured, inject these delays (with some domain randomization) into microban_constants.py (actuators) and microban_velocity_env_cfg.py (observations).

---

BAM Actuator Identification

The recently updated BAM documentation covers the entire identification procedure in depth. Please refer to it for the full process: BAM Documentation

To answer your specific points:

1. Conditions: Use the trajectories provided in the documentation. The operating mode during the test must be the exact same one used on the real robot.
2. Settings to record: You need to record the parameters that are defined in the CMA-ES optimization.
3. What BAM models: I think that BAM models all the behaviors you cited.
4. Delays: Communication and actuator delays must be modeled separately. Delays measured on a minimal test bench are different from the ones on the fully assembled real robot's bus.
5. Loads: Please check the documentation for the setup. The test pendulum is never recorded entirely unloaded.
6. Validation: Keep a separate validation dataset or use different trajectories on the pendulum.
7. Metrics: I typically use MAE. There is no clear threshold for walking, but the goal is to observe an error reduction compared to the default BAM models and to succeed in sim-to-sim first. If sim-to-sim works but sim-to-real fails, the issue is likely in the checklist above.

[xsatvik]

Really enjoyed the conversation here, any approach to handle the servo backlash?
Because if i do sysid of a hanging robot, its quite a difference compared for it to be on the ground where i observe significant backlash as well.

thanks!

[MarcDcls]

Hi @xsatvik,

​Currently, BAM does not perform backlash identification. If you plan to identify it yourself, I recommend testing several motors with varying levels of wear, since backlash generally changes throughout a motor's lifespan.

​However, one way to account for this phenomenon during training is to add a passive DoF at each joint after the active joint, limiting it to a few degrees based on the backlash value you identified. If you take this approach, I recommend randomizing this range across different motors and environments.

​Hope this helps,

​Marc