simplify list_data_collate and collate_meta_tensor using collate

[wyli]

the generic collate API is implemented in pytorch

pytorch/pytorch#85748

def collate(batch, *, collate_fn_map: Optional[Dict[Union[Type, Tuple[Type, ...]], Callable]] = None):
    r"""
        General collate function that handles collection type of element within each batch
        and opens function registry to deal with specific element types. `default_collate_fn_map`
        provides default collate functions for tensors, numpy arrays, numbers and strings.

probably the monai helper functions could be simplified:

MONAI/monai/data/utils.py

Lines 448 to 451 in 50c3f32

	def list_data_collate(batch: Sequence):
	"""
	Enhancement for PyTorch DataLoader default collate.
	If dataset already returns a list of batch data that generated in transforms, need to merge all data to 1 list.

MONAI/monai/data/utils.py

Lines 427 to 430 in 50c3f32

	def collate_meta_tensor(batch):
	"""collate a sequence of meta tensor sequences/dictionaries into
	a single batched metatensor or a dictionary of batched metatensor"""
	if not isinstance(batch, Sequence):

[Nic-Ma]

It may also affect this feature request #5909?
CC @yiheng-wang-nv

[kbressem]

It would be cool if in this contribution, the collate_meta_tensor could be more permissive, if keys in the metadata are missing. I work with data from multiple datasets and frequently get errors, as keys are not the same across the different images. I usually just pop these keys in the metadata, but it would be nice, if this would not be an error at all.

  File "/home/bressekk/miniconda3/envs/monailabel/lib/python3.9/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/bressekk/miniconda3/envs/monailabel/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 61, in fetch
    return self.collate_fn(data)
  File "/home/bressekk/miniconda3/envs/monailabel/lib/python3.9/site-packages/monai/data/utils.py", line 470, in list_data_collate
    ret = collate_meta_tensor(data)
  File "/home/bressekk/miniconda3/envs/monailabel/lib/python3.9/site-packages/monai/data/utils.py", line 439, in collate_meta_tensor
    return {k: collate_meta_tensor([d[k] for d in batch]) for k in elem_0}
  File "/home/bressekk/miniconda3/envs/monailabel/lib/python3.9/site-packages/monai/data/utils.py", line 439, in <dictcomp>
    return {k: collate_meta_tensor([d[k] for d in batch]) for k in elem_0}
  File "/home/bressekk/miniconda3/envs/monailabel/lib/python3.9/site-packages/monai/data/utils.py", line 434, in collate_meta_tensor
    collated.meta = default_collate([i.meta or TraceKeys.NONE for i in batch])
  File "/home/bressekk/miniconda3/envs/monailabel/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 265, in default_collate
    return collate(batch, collate_fn_map=default_collate_fn_map)
  File "/home/bressekk/miniconda3/envs/monailabel/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 128, in collate
    return elem_type({key: collate([d[key] for d in batch], collate_fn_map=collate_fn_map) for key in elem})
  File "/home/bressekk/miniconda3/envs/monailabel/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 128, in <dictcomp>
    return elem_type({key: collate([d[key] for d in batch], collate_fn_map=collate_fn_map) for key in elem})
  File "/home/bressekk/miniconda3/envs/monailabel/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 128, in <listcomp>
    return elem_type({key: collate([d[key] for d in batch], collate_fn_map=collate_fn_map) for key in elem})
KeyError: 'ITK_non_uniform_sampling_deviation'

[Nic-Ma]

Hi @wyli ,

Could you please help double confirm it?

Thanks in advance.

[wyli]

created an additional feature request to make sure the requirement tracked