panda-gym v2

[qgallouedec]

Description

Update benchmark for latest version of panda-gym.

Motivation and Context

• I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (update in the documentation)

Checklist:

• I've read the CONTRIBUTION guide (required)
• I have updated the changelog accordingly (required).
• My change requires a change to the documentation.
• I have updated the tests accordingly (required for a bug fix or a new feature).
• I have updated the documentation accordingly.
• I have reformatted the code using make format (required)
• I have checked the codestyle using make check-codestyle and make lint (required)
• I have ensured make pytest and make type both pass. (required)

Note: we are using a maximum length of 127 characters per line

[araffin]

Hey,

thanks for the PR =)
I might wait until Huggingface PR is merged #198
I will help @simoninithomas in the coming days.

[araffin]

Quickly looking at it, no additional tuning was in fact required?

[qgallouedec]

no additional tuning was in fact required?

No tuning has been done. It might be possible to find better hyperparameters. I leave this for a future PR.
One exception: I had to decrease the learning for PickAndPlace with SAC. Otherwise, the loss was diverging.

[araffin]

Basic version is ready to review ;)
#250

An example: https://huggingface.co/sb3/ppo-MountainCarContinuous-v0

[araffin]

btw, this requires gym > 0.21, right?

[qgallouedec]

btw, this requires gym > 0.21, right?

Yes, so it can't be merged before #256 I think

[fikricanozgur]

What's the status with this PR?

[qgallouedec]

panda-gym v2 requires gym==0.24. But, for the moment, SB3 requires gym==0.21. So the status is "waiting for SB3 to be compatible with gym>=0.24"