Skip to content
This repository was archived by the owner on Nov 15, 2022. It is now read-only.
This repository was archived by the owner on Nov 15, 2022. It is now read-only.

Cannot use nestedtensor in Dataloader? #14

@justanhduc

Description

@justanhduc

I got a pickler error in multiprocessing that it cant pickle the nestedtensor object. does it mean i cant use nestedtensor for Dataset?
I guess using nestedtensor in Dataset is important as nestedtensor always copies tensors, which will double the memory usage.

Traceback (most recent call last):
  File "/home/justanhduc/anaconda3/lib/python3.7/multiprocessing/queues.py", line 236, in _feed
    obj = _ForkingPickler.dumps(obj)
  File "/home/justanhduc/anaconda3/lib/python3.7/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
TypeError: can't pickle nestedtensor._C._BufferNestedTensor objects

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions