Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

requires_grad attribute for meta tensors #96

Closed
2 tasks
vmoens opened this issue Apr 26, 2022 · 0 comments · Fixed by #135
Closed
2 tasks

requires_grad attribute for meta tensors #96

vmoens opened this issue Apr 26, 2022 · 0 comments · Fixed by #135
Labels
enhancement New feature or request

Comments

@vmoens
Copy link
Contributor

vmoens commented Apr 26, 2022

MetaTensors is a special TorchRL class that tracks the metadata of tensors stored in the TensorDict.
Its main reason of existence is to avoid the overhead of loading tensors just to read an attribute. We already have a SavedTensordict class, for instance, that represents tensors saved on disk. If we had to load the whole tensordict from disk every time an attribute (shape, etc.) is gathered this could create an unecessary overhead.
We carefully choose which attribute should be part of MetaTensors. Some may not be easy to infer and are therefore not part of the metatensor.
It may be useful to implement a requires_grad attribute in MetaTensor. Its purpose would simply be to tell if a tensor is part of a graph or not.

In the tests, one should make sure that these aspects are accounted for:

  • For LazyStackTensorDict objects, the resulting tensors will require a gradient if a single tensor of the list requires a gradient. The respective MetaTensor should follow the same logic.
  • In theory, MemmapTensors and SavedTensorDicts are not compatible with gradients that are part of computational graphs. This means that when writing a tensor that has requires_grad in a SavedTensorDict, an error should be raised. Same if a regular TensorDict is cast using tensordict.to(SavedTensorDict) or tensordict.memmap_().
@vmoens vmoens added the enhancement New feature or request label Apr 26, 2022
kovalevfm added a commit to kovalevfm/rl that referenced this issue May 16, 2022
Added requires_grad to MetaTensor and some checks to prevent rquires_grad tensors to be converted into SavedTenosrDict and MemmapTensor
@vmoens vmoens linked a pull request May 16, 2022 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant