Skip to content

Conversation

@matteobettini
Copy link
Member

@matteobettini matteobettini commented Jan 8, 2024

New features

  • The actions agent.action.u (which before corresponed to agent.state.force) are now detached from the physics. Dynamics will read actions and write into agent.state.force and agent.state.torque. This allows users to play around with action sizes and new dynamics more flexibly.
  • At agent construction you can specify the agent dynamics (Agent(dynamics=Holonomic()) is the default). There are many available (Holonomic,HolonomicWithRotation,DiffDrive,KinematicBicycle)
  • The dynamics will define the action size. You can also override this and define an action size yourself Agent(action_size=3). But you have to provide at least the minimum number of actions required by the dynamics (dynamics.needed_action_size). The dynamics will use the first dynamics.needed_action_size actions and the rest can be used for other user needs
  • The discrete actions will have 3 options for each agent.action_size action. By default the holonomic agent has 2 actions: force_x (with 3 options stay,decrement,increment) and force_y (with 3 options stay,decrement,increment). Holonomic agent with rotations have 3 actions, with the additional one controlling torque (with the 3 options stay, decrement, increment). And so on for other dynamics....
  • Action ranges, multipliers and noise can be controlled at agent construction with u_range, u_multiplier and u_noise. These can either be a scalar or a list. In case of a list, the list must have length agent.action_size

BC-BREAKING Changes

  • Removed u_rot_range,u_rot_multiplier in favor of a more general u_range, u_multiplier and u_noise which can be a list (one elem per action )
  • Removed MultiDiscrete actions. While before in case of multiple actions there was a space Multidiscrete([action_1_size, action_2_size]) , now there is a space DIscrete(action_1_size * action_2_size). It is still possible to ask MultiDIscrete action spaces by setting vmas.make_env(multidiscrete_actions=True)
  • Discrete actions for the default dynamics (holonomic no rotation) have changed from 5 (up,down,left,right,stay) to 9, that now include the corners.

FIxes #51
Fixes #75

cc @Jianye-Xu @Zartris

@matteobettini matteobettini changed the title [Feature] Actions detched from physics and any number of actions [Feature] Actions detached from physics and any number of actions Jan 8, 2024
@matteobettini matteobettini changed the title [Feature] Actions detached from physics and any number of actions [Feature] Actions detached from physics and allow any number of actions Jan 8, 2024
@matteobettini
Copy link
Member Author

@Jianye-Xu the pr should be functional, feel free to try it out and let me know if you have some feedback

@matteobettini matteobettini marked this pull request as ready for review January 8, 2024 17:12
@Jianye-Xu
Copy link
Contributor

@matteobettini Hello Matteo, thanks for your work! I mainly checked the kinematic bicycle part, and it works well. I think you can close #75 now.

@Jianye-Xu
Copy link
Contributor

A small feedback: in use_vmas_env.py, the default value of variable simple_2d_action is [0, -1.0] if continuous_actions = True. This would cause an error if you set scenario_name to kinematic_bicycle, in which the u_range is set such that the maximum allowed steering angle is 30 degree (around 0.524 in radius). To avoid this error, the absolute value of the second entry of simple_2d_action should be smaller than 0.524. For example, setting [0, -0.5] as the default value for simple_2d_action could avoid the error.

@matteobettini
Copy link
Member Author

A small feedback: in use_vmas_env.py, the default value of variable simple_2d_action is [0, -1.0] if continuous_actions = True. This would cause an error if you set scenario_name to kinematic_bicycle, in which the u_range is set such that the maximum allowed steering angle is 30 degree (around 0.524 in radius). To avoid this error, the absolute value of the second entry of simple_2d_action should be smaller than 0.524. For example, setting [0, -0.5] as the default value for simple_2d_action could avoid the error.

Got it. Will make sure the simple actions are feasible in all scenarios

@matteobettini matteobettini merged commit f09a406 into main Jan 10, 2024
@matteobettini matteobettini deleted the any_actions branch January 10, 2024 08:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Issue] Unmatched action size for kinematic bicycle model and differential drive robots [Feature] Allow any number of actions

3 participants