Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature]: TensorDictPrimer transform #456

Merged
merged 16 commits into from
Sep 23, 2022

Conversation

nicolas-dufour
Copy link
Contributor

@nicolas-dufour nicolas-dufour commented Sep 15, 2022

Description

This PR add a ForceTensorReset transform.

Motivation and Context

ForceTensorReset allows to set or reset to default values some given vectors

Types of changes

What types of changes does your code introduce? Remove all that do not apply:

  • New feature (non-breaking change which adds core functionality)

Checklist

Go over all the following points, and put an x in all the boxes that apply.
If you are unsure about any of these, don't hesitate to ask. We are here to help!

  • I have read the CONTRIBUTION guide (required)
  • My change requires a change to the documentation.
  • I have updated the tests accordingly (required for a bug fix or a new feature).
  • I have updated the documentation accordingly.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 15, 2022
@vmoens
Copy link
Contributor

vmoens commented Sep 15, 2022

Lint!

@vmoens
Copy link
Contributor

vmoens commented Sep 16, 2022

ForceTensorReset is not very indicative IMO
What about

  • DefaultTensorRest
  • DefaultValueReset
  • TensorDictPrimerReset
  • TensorDictPrimer
  • TensorDictPrimerTransform

@vmoens
Copy link
Contributor

vmoens commented Sep 16, 2022

Added TensorDictPrimer
Let me know if that's a suitable solution. The big advantage is that the spec is registered in the env, and we can execute rollouts in parallel without problem as the tensordict will be properly pre-allocated.
If you're fine with it, can you remove ForceTensorReset as well as the tests?
You can rename the class if you feel like it :)

@vmoens vmoens added the enhancement New feature or request label Sep 16, 2022
@nicolas-dufour
Copy link
Contributor Author

The transform is still needed to properly reset the states. Thank you for the changes to rollout!. I'll rename it to DefaultTensorReset if that's ok with you

@vmoens
Copy link
Contributor

vmoens commented Sep 16, 2022

Have you looked at my solution? It does reset things. What does ForceTensorReset do that TensorDictPrimer does not?

@nicolas-dufour
Copy link
Contributor Author

I'm not sure i understand how i can set vectors that aren't given by the env with TensorDictPrimer. Like prior_state and belief

@vmoens
Copy link
Contributor

vmoens commented Sep 16, 2022

Look at the docstring :)

@nicolas-dufour
Copy link
Contributor Author

Ah it's with the kwargs. Ok i will remove my tests and transform then

@vmoens vmoens changed the title [Feature]: Added class ForceTensorReset transform [Feature]: TensorDictPrimer transform Sep 23, 2022
@vmoens vmoens merged commit c498aac into pytorch:main Sep 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants