TensorDict v0.14.0

TensorDict v0.14.0

Highlights

TensorDict 0.14.0 introduces Zarr-backed persistent storage, portable single-file memory-mapped archives, and direct autograd through TensorDict.backward(). It also improves serialization performance, distributed point-to-point operations, UnbatchedTensor interoperability, and memmap correctness and security.

Breaking Changes

• TensorDict.to_module() now preserves existing module parameter and buffer registrations by default. Pass preserve_module_state=False to retain the previous replacement behavior (#1720).
• TensorDict.copy_at_() now uses its optimized tensor-only path by default. Pass fast=False to retain the previous update_at_() fallback behavior (#1691).

Deprecations

• Memory-mapped loading now accepts allow_pickle across load and refresh APIs. In 0.14, omitting it still loads pickled non-tensor data with a warning; the default will become allow_pickle=False in 0.15. Pass True only for trusted artifacts or False to reject pickle now (#1763).

Features

• Added Zarr-backed PersistentTensorDict storage through to_zarr(), from_zarr(), and compatible store backends (#1754).
• Added portable single-file .tdz archives for memory-mapped TensorDicts, including lazy loading, subpath access, compression, writable mappings, and pack/unpack utilities (#1735).
• Added TensorDict.backward() for differentiating all compatible tensor leaves in one autograd operation (#1733).
• Added group-relative peers through group_dst and group_src for distributed send and receive operations (#1739).
• Added vmap support and Python scalar conversions for UnbatchedTensor (#1729, #1728).
• Added early-return support to multithreaded TensorDict consolidation.

Bug Fixes

• Converted mapping values into TensorDict values for TensorDict-typed fields in tensor-only tensorclasses (#1753).
• Hardened memmap paths against symlink traversal and unsafe output targets (#1762).
• Propagated dataset-creation options through nested persistent TensorDicts (#1759).
• Preserved zero-element tensors during memmap round trips (#1750).
• Fixed consolidation of non-contiguous TensorDicts (#1747).
• Preserved NonTensorStack entries in contiguous() (#1746).
• Allowed TensorDictParams.load_state_dict(strict=False) to skip missing submodules (#1734).
• Fixed _SubTensorDict memmap round trips (#1736).
• Preserved lazy concatenation for mixed TensorDict inputs (#1732).
• Kept recursive TensorDict metadata consistent and corrected UnbatchedTensor stack metadata (#1731, #1730).
• Preserved non-tensor entries during padding and boolean-mask indexing (#1725).
• Preserved TensorClass subclass identity across memmap save/load operations (#1722).

Performance

• Reused a shared worker pool across serialization operations.
• Removed a multithreaded consolidation slowdown caused by chunked copies.
• Improved Zarr metadata handling and lazy-read performance.

Contributors

Thanks to @peterdsharpe, and @javierdejesusda, both first-time contributors!

Full Changelog: v0.13.0...v0.14.0