Skip to content

refactor!: unify env/scenes API across robots and restructure assets#283

Merged
juelg merged 99 commits intomasterfrom
feat/unified-env-config-and-assets-refactor
Apr 23, 2026
Merged

refactor!: unify env/scenes API across robots and restructure assets#283
juelg merged 99 commits intomasterfrom
feat/unified-env-config-and-assets-refactor

Conversation

@juelg
Copy link
Copy Markdown
Contributor

@juelg juelg commented Apr 22, 2026

Summary

This PR is a large refactor that brings the environment/scenes stack to a unified config + creator model across robots, updates examples/docs accordingly, and restructures the assets layout for cleaner composition and reuse.

What Changed

Core API and Architecture

  • Refactored env/scenes APIs toward stateless, config-driven creation.
  • Added/expanded new env modules:
    • python/rcs/envs/configs.py
    • python/rcs/envs/scenes.py
    • python/rcs/envs/sim.py
    • task helpers and task modules
  • Removed legacy creator path:
    • python/rcs/envs/creators.py (deleted)
  • Updated Python exports, typing stubs, and bindings to match new interfaces.

Robot Extensions

  • Added/updated per-robot config + creator integration for FR3, Panda, UR5e, XArm7, and SO101.
  • Updated hardware env creation flows to use the new RCSEnvCreator + config approach.
  • Cleaned up old default factories/creator utilities.

Simulation and Composer

  • Extended scene composer capabilities (objects, cameras, frame objects, offsets, gravcomp handling).
  • Added/fixed camera support and related sim-camera behavior (including memory leak fix via ring buffer).
  • Updated robot/gripper integration points and related C++/pybind surfaces.

Assets and Scene Layout

  • Major asset restructuring under robot/gripper/camera/object folders.
  • Added/updated multiple asset XMLs/meshes and scene references.
  • Removed legacy generated/duplicated scene-embedded assets.

Examples, Tests, and Docs

  • Updated FR3/UR5e/XArm7/SO101 and teleop examples to new API.
  • Updated tests for scene/env refactor.
  • Refreshed user docs/getting started/extension docs.
  • Final lint/format/mypy cleanup commits included.

Breaking Changes

  • Legacy env creator flow was removed/replaced (python/rcs/envs/creators.py).
  • Robot/sim configuration expectations changed (object-based robot types and updated scene/composer config model).
  • Task-specific object pathways include removals/replacements (e.g. ParallelPickTask path changes).

Notes for Review

  • This PR is broad (API + assets + examples/docs).
  • Best reviewed by area:
    1. python/rcs/envs/* and python/rcs/sim/composer.py
    2. extensions/rcs_*/*
    3. assets restructuring
    4. examples/docs/tests alignment

juelg added 30 commits April 12, 2026 12:57
@juelg
refactor(assets): restructure assets folder
566423c 
- restructure assets: clearer structure and only contain absolute minimum for RCS examples
- updated actuators in so101
- reset ur5e, fr3 and panda to original menagerie xml
- added gravcomp script that can add and remove gravity compensation
- removed gravity compensation of all xmls as this will be done in the composer
@juelg
chore: remove unused left over of collision guard
d4a107f
@juelg
refactor!: pinocchio default for urdf is now false
75ecfbb
@juelg
refactor(sim): postfix id is now prefix for better mjspec logic
1ce8913
@juelg
feat(sim): add inital mjspec composer version
d822b11
@juelg
Merge branch 'master' into juelg/refactor-assets
9ffe74c
@juelg
refactor(sim)!: remove mjcf_scene_path from robot config
5ca3d84
@juelg
refactor!: robot type now object based
eac502b 
- removed meta robot config due to limited extensibility
- moved q_home and joint_limits to robot config
- refactored robot type: now class based for python extensibility
- readded wrapper configs: home_on_reset for robot wrapper and binary from gripper wrapper, they need to be extra configs for composition and not part of the robot/gripper config
@juelg
feat: scenes api
4535d95 
- added scenes api
- fixed quaternion bug in composer
- fixed bug in add prefix of sim robot config in attachment site
- kinematic file is no longer default urdf but mjcf
- added panda to robot type
- SimRobotConfig: added missing dof, joint_limits and q_home
@juelg
feat(composer): gravity compensation
3eaa31a 
- add gravity compensation for joints and bodies
- add gravity compensation in scene api
- fix: integrator implicit fast in scene xml
@juelg
chore: remove legacy scenes definition
06658fa
@juelg
style: formatting
dc93a8c
@juelg
lint: python linting fixes
484c2d0
@juelg
feat: added readded robot meta config in python
e385ebb
@juelg
test: fix kinematic test
9de3c26
@juelg
feat(scenes): seperate robot mjcf model
8c5ee25 
- added scene paths dict
- moved attachment site to add gripper in model composer
- simscene: moved mjcf composition to load_scene
- simscene: added option to load seperate robot mjcf different from robot config, this allows for example to have two arms which are represented in one model
@juelg
feat: proper frame definitions in sim scene config
5d2bab0
@juelg
feat(sim scene): supports objects
67be1a6
@juelg
style: fix linting errors
22a603a
@juelg
feat(sim scene): prefix cfg accessible
b234b20
@juelg
feat(composer): add camera function
0847af6
@juelg
feat(sim scene): adding cameras by config
f3025e4 
- fix: action and observation space exposing in multi robot wrapper
- fix: sim camera config was missing copy and was returning base config when deepcopy
- feat: added camera adding to sim scene api
- example: added cameras to the scene.py fr3 empty world example
@juelg
fix(sim): robotiq stl
ea2515e
@juelg
fix: dof in metaconfig of ur and so
f7a83f9
@juelg
fix(bindings): add missing gripper type
b50d8d9
@juelg
feat(sim): ur5e in scene composer
3b375bb
@juelg
fix: frame rate limiter
79af71d
@juelg
feat(sim): gripper access fix
86b94e9
@juelg
examples: fix fr3 examples to follow new api
60bea98
@juelg
feat(composer): add optional gripper offset
04add60
juelg added 2 commits April 22, 2026 18:19
@juelg
lint: fix ruff linting errors
4932b6e 
- also fixed camera config in franka.py teleop script
@juelg
lint: fix mypy linting
e0e9014
@juelg juelg mentioned this pull request Apr 22, 2026
Closed
@juelg juelg force-pushed the feat/unified-env-config-and-assets-refactor branch 6 times, most recently from 39c7d18 to 2b4898b Compare April 22, 2026 20:31
@juelg juelg force-pushed the feat/unified-env-config-and-assets-refactor branch from 2b4898b to 02497d2 Compare April 22, 2026 20:31
@juelg juelg mentioned this pull request Apr 23, 2026
Open
@juelg juelg force-pushed the feat/unified-env-config-and-assets-refactor branch from 330e711 to 9536298 Compare April 23, 2026 05:17
@juelg juelg merged commit 88594d3 into master Apr 23, 2026
13 checks passed
This was referenced Apr 23, 2026
Open
Closed
Closed
Closed
Closed
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@juelg @tenfoldpaper @Simonhil