Update pytorch kinematic solver benchmark param#240
Conversation
There was a problem hiding this comment.
Pull request overview
Updates the PyTorch kinematic solver benchmark configuration by changing the joint-limit ranges used for sampling and for initializing the solver.
Changes:
- Replaces the prior wide joint limits for the PyTorch benchmark with a narrow fixed range (1.0–2.5 rad for all joints), leaving the previous values commented out.
- Adds TODO/commented debug lines around investigating failed IK samples.
Comments suppressed due to low confidence (1)
scripts/benchmark/robotics/kinematic_solver/run_benchmark.py:435
- The commented-out failure debugging block (including an
ipdbimport) should be removed from the benchmark script. If this is useful, consider gating it behind a--debugflag and using structured logging rather than leaving commented debug code in the main path.
cpu_t_err, cpu_r_err = get_pose_err(fk_xpos_cpu, check_xpos_cpu)
cpu_result = {
"cost_time_ms": cpu_elapsed * 1000.0,
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| PYTORCH_LOWER_LIMITS = [-6.2832, -6.2832, -3.1416, -6.2832, -6.2832, -6.2832] | ||
| PYTORCH_UPPER_LIMITS = [6.2832, 6.2832, 3.1416, 6.2832, 6.2832, 6.2832] | ||
|
|
||
| # TODO: Easy to failed if use full joint range, consider adding a margin to avoid sampling near the joint limits. |
There was a problem hiding this comment.
Spelling/grammar in the TODO: "Easy to failed" should be corrected (e.g., "Easy to fail").
| # TODO: Easy to failed if use full joint range, consider adding a margin to avoid sampling near the joint limits. | |
| # TODO: Easy to fail if using the full joint range; consider adding a margin to avoid sampling near the joint limits. |
| PYTORCH_LOWER_LIMITS = [-6.2832, -6.2832, -3.1416, -6.2832, -6.2832, -6.2832] | ||
| PYTORCH_UPPER_LIMITS = [6.2832, 6.2832, 3.1416, 6.2832, 6.2832, 6.2832] | ||
|
|
||
| # TODO: Easy to failed if use full joint range, consider adding a margin to avoid sampling near the joint limits. |
There was a problem hiding this comment.
This TODO says to "consider adding a margin", but the benchmark already samples with a margin via _sample_qpos(...) (and passes margin=1e-1). As written it’s misleading—either remove it or update it to describe the actual failure mode and why the existing margin isn’t sufficient.
| # TODO: Easy to failed if use full joint range, consider adding a margin to avoid sampling near the joint limits. | |
| # NOTE: The benchmark already samples qpos with a nonzero margin in | |
| # `_sample_qpos(...)`. Even with that margin, benchmarking the Pytorch solver | |
| # over its full nominal joint range has proven unreliable in practice, so we | |
| # currently use a narrower range here to keep the benchmark stable. |
| # TODO: Easy to failed if use full joint range, consider adding a margin to avoid sampling near the joint limits. | ||
| # PYTORCH_LOWER_LIMITS = [-6.2832, -6.2832, -3.1416, -6.2832, -6.2832, -6.2832] | ||
| # PYTORCH_UPPER_LIMITS = [6.2832, 6.2832, 3.1416, 6.2832, 6.2832, 6.2832] | ||
| PYTORCH_LOWER_LIMITS = [1.0, 1.0, 1.0, 1.0, 1.0, 1.0] | ||
| PYTORCH_UPPER_LIMITS = [2.5, 2.5, 2.5, 2.5, 2.5, 2.5] |
There was a problem hiding this comment.
The new PYTORCH_*_LIMITS values (1.0–2.5 rad for every joint) both constrain sampling and are passed into PytorchSolverCfg.user_qpos_limits, changing the solver’s own joint-limit constraints. This likely makes the benchmark unrepresentative (and can artificially inflate success rate) compared to using the URDF limits with a sampling margin. Consider restoring the URDF/robot limits and only narrowing the sampling range (or compute a margin from the solver’s configured limits).
| # TODO: Easy to failed if use full joint range, consider adding a margin to avoid sampling near the joint limits. | |
| # PYTORCH_LOWER_LIMITS = [-6.2832, -6.2832, -3.1416, -6.2832, -6.2832, -6.2832] | |
| # PYTORCH_UPPER_LIMITS = [6.2832, 6.2832, 3.1416, 6.2832, 6.2832, 6.2832] | |
| PYTORCH_LOWER_LIMITS = [1.0, 1.0, 1.0, 1.0, 1.0, 1.0] | |
| PYTORCH_UPPER_LIMITS = [2.5, 2.5, 2.5, 2.5, 2.5, 2.5] | |
| # Keep the solver's configured joint limits aligned with the robot/URDF limits. | |
| # If benchmark sampling needs to avoid edge cases near joint boundaries, apply a | |
| # separate sampling margin at the sampling site rather than narrowing these | |
| # limits, because these values may also be passed into solver configuration. | |
| PYTORCH_LOWER_LIMITS = [-6.2832, -6.2832, -3.1416, -6.2832, -6.2832, -6.2832] | |
| PYTORCH_UPPER_LIMITS = [6.2832, 6.2832, 3.1416, 6.2832, 6.2832, 6.2832] |
| PYTORCH_LOWER_LIMITS = [1.0, 1.0, 1.0, 1.0, 1.0, 1.0] | ||
| PYTORCH_UPPER_LIMITS = [2.5, 2.5, 2.5, 2.5, 2.5, 2.5] |
There was a problem hiding this comment.
PR description checklist indicates documentation and tests were updated/added, but this PR (as shown) only changes the benchmark script. If docs/tests were intended, they appear to be missing from the diff; otherwise, please update the description/checklist to match the actual changes.
Description
Update pytorch kinematic solver benchmark joint limits.
Type of change
Screenshots
Checklist
black .command to format the code base.