Skip to content

Multiple Instance Learning (MIL) Examples RuntimeError: LoadImage cannot find a suitable reader for file #844

@DocandBean

Description

@DocandBean

Describe the bug
Command

python -u panda_mil_train_evaluate_pytorch_gpu.py --data_root=/PandaChallenge2020/train_images --amp --distributed --mil_mode=att_trans --batch_size=4 --epochs=50 --logdir=./logs

generates

RuntimeError: LoadImage cannot find a suitable reader for file: /PandaChallenge2020/train_images/c2b4cc586b3984b248e7d095d4c0bd03.tiff.

Full error message below

To Reproduce
Steps to reproduce the behavior:

  1. git clone MONAI tutorials
  2. Install suggested required dependencies tifffile, imagecodecs (via pip) and downloaded Kaggle file per README
  3. Install sklearn via pip
  4. Run command 'python -u panda_mil_train_evaluate_pytorch_gpu.py --data_root=/PandaChallenge2020/train_images --amp --distributed --mil_mode=att_trans --batch_size=4 --epochs=50 --logdir=./logs'

Expected behavior
Train in multi-gpu mode with AMP using all available gpus, assuming the training images in /PandaChallenge2020/train_images folder, using pre-defined 80/20 data split

Environment (please complete the following information):

  • OS: Ubuntu 20.04.4 LTS
  • Python version: 3.9
  • MONAI version: 0.9.1+14.g81fb7e41
  • CUDA/cuDNN version: 11.6.0
  • GPU models and configuration: NVIDIA A100

Additional context
Access/permissions to the file and the ImageLoader class do not seem to be the problem.

import os
import shutil
from monai.transforms import LoadImage
from monai.config import print_config

filename = 'PandaChallenge2020/train_images/c2b4cc586b3984b248e7d095d4c0bd03.tiff'

data, meta = LoadImage()(filename)

print(f"image data shape:{data.shape}")
print(f"meta data:{meta}")

from monai.data import WSIReader

data, meta = LoadImage(WSIReader)(filename)

print(f"image data shape:{data.shape}")
print(f"meta data:{meta}")

yields the following output

image data shape:(7424, 8448, 4)
meta data:{'spacing': array([0.00048619, 0.00048619]), original_affine: array([[-4.86187619e-04,  0.00000000e+00,  0.00000000e+00],
       [ 0.00000000e+00, -4.86187619e-04,  0.00000000e+00],
       [ 0.00000000e+00,  0.00000000e+00,  1.00000000e+00]]), space: RAS, affine: tensor([[-4.8619e-04,  0.0000e+00,  0.0000e+00],
        [ 0.0000e+00, -4.8619e-04,  0.0000e+00],
        [ 0.0000e+00,  0.0000e+00,  1.0000e+00]], dtype=torch.float64), spatial_shape: array([7424, 8448]), original_channel_dim: -1, 'filename_or_obj': 'PandaChallenge2020/train_images/c2b4cc586b3984b248e7d095d4c0bd03.tiff'}
image data shape:(3, 8448, 7424)
meta data:{'backend': 'cucim', 'original_channel_dim': 0, 'spatial_shape': array([8448, 7424]), 'num_patches': 1, 'path': '/home/dabean/MONAI_tutorials/tutorials/pathology/multiple_instance_learning/PandaChallenge2020/train_images/c2b4cc586b3984b248e7d095d4c0bd03.tiff', 'patch_location': array([0, 0]), 'patch_size': array([8448, 7424]), 'patch_level': 0, 'filename_or_obj': 'PandaChallenge2020/train_images/c2b4cc586b3984b248e7d095d4c0bd03.tiff', affine: tensor([[1., 0., 0., 0.],
        [0., 1., 0., 0.],
        [0., 0., 1., 0.],
        [0., 0., 0., 1.]], dtype=torch.float64), space: RAS}

Complete Error Output

python -u panda_mil_train_evaluate_pytorch_gpu.py --data_root=/PandaChallenge2020/train_images --amp --distributed --mil_mode=att_trans --batch_size=4 --epochs=50 --logdir=./logs
Argument values:
data_root => /PandaChallenge2020/train_images
dataset_json => None
num_classes => 5
mil_mode => att_trans
tile_count => 44
tile_size => 256
checkpoint => None
validate => False
logdir => ./logs
epochs => 50
batch_size => 4
optim_lr => 3e-05
weight_decay => 0
amp => True
val_every => 1
workers => 2
distributed => True
world_size => 1
rank => 0
dist_url => tcp://127.0.0.1:23456
dist_backend => nccl
quick => False
-----------------
Multigpu 1 rescaled lr 1.5e-05
0  gpu 0
Batch size is: 4 epochs 50
Dataset training: 8490 validation: 2124
The parameter 'pretrained' is deprecated since 0.13 and will be removed in 0.15, please use 'weights' instead.
Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and will be removed in 0.15. The current behavior is equivalent to passing `weights=ResNet50_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet50_Weights.DEFAULT` to get the most up-to-date weights.
Writing Tensorboard logs to  ./logs
0 Wed Aug  3 17:53:21 2022 Epoch: 0
Traceback (most recent call last):
  File "/home/dabean/MONAI_tutorials/tutorials/pathology/multiple_instance_learning/panda_mil_train_evaluate_pytorch_gpu.py", line 547, in <module>
    mp.spawn(main_worker, nprocs=ngpus_per_node, args=(args,))
  File "/home/dabean/miniconda3/envs/MONAI_d01_env/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 240, in spawn
    return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
  File "/home/dabean/miniconda3/envs/MONAI_d01_env/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 198, in start_processes
    while not context.join():
  File "/home/dabean/miniconda3/envs/MONAI_d01_env/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 160, in join
    raise ProcessRaisedException(msg, error_index, failed_process.pid)
torch.multiprocessing.spawn.ProcessRaisedException:

-- Process 0 terminated with the following error:
Traceback (most recent call last):
  File "/home/dabean/miniconda3/envs/MONAI_d01_env/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 69, in _wrap
    fn(i, *args)
  File "/home/dabean/MONAI_tutorials/tutorials/pathology/multiple_instance_learning/panda_mil_train_evaluate_pytorch_gpu.py", line 414, in main_worker
    train_loss, train_acc = train_epoch(model, train_loader, optimizer, scaler=scaler, epoch=epoch, args=args)
  File "/home/dabean/MONAI_tutorials/tutorials/pathology/multiple_instance_learning/panda_mil_train_evaluate_pytorch_gpu.py", line 48, in train_epoch
    for idx, batch_data in enumerate(loader):
  File "/home/dabean/miniconda3/envs/MONAI_d01_env/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 652, in __next__
    data = self._next_data()
  File "/home/dabean/miniconda3/envs/MONAI_d01_env/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1347, in _next_data
    return self._process_data(data)
  File "/home/dabean/miniconda3/envs/MONAI_d01_env/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1373, in _process_data
    data.reraise()
  File "/home/dabean/miniconda3/envs/MONAI_d01_env/lib/python3.9/site-packages/torch/_utils.py", line 461, in reraise
    raise exception
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/dabean/MONAI_install/MONAI/monai/transforms/transform.py", line 90, in apply_transform
    return _apply_transform(transform, data, unpack_items)
  File "/home/dabean/MONAI_install/MONAI/monai/transforms/transform.py", line 54, in _apply_transform
    return transform(parameters)
  File "/home/dabean/MONAI_install/MONAI/monai/transforms/io/dictionary.py", line 133, in __call__
    data = self._loader(d[key], reader)
  File "/home/dabean/MONAI_install/MONAI/monai/transforms/io/array.py", line 245, in __call__
    raise RuntimeError(
RuntimeError: LoadImage cannot find a suitable reader for file: /PandaChallenge2020/train_images/c2b4cc586b3984b248e7d095d4c0bd03.tiff.
    Please install the reader libraries, see also the installation instructions:
    https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies.
   The current registered: [<monai.data.image_reader.PydicomReader object at 0x7f0d64ae22b0>, <monai.data.image_reader.ITKReader object at 0x7f0c56023940>, <monai.data.image_reader.NrrdReader object at 0x7f0c56023970>, <monai.data.image_reader.NumpyReader object at 0x7f0c56023a00>, <monai.data.image_reader.PILReader object at 0x7f0c55bc9250>, <monai.data.image_reader.NibabelReader object at 0x7f0c55bc9340>, <monai.data.wsi_reader.WSIReader object at 0x7f0c55bc93d0>].
Traceback (most recent call last):
  File "/home/dabean/MONAI_install/MONAI/monai/transforms/io/array.py", line 230, in __call__
    img = reader.read(filename)
  File "/home/dabean/MONAI_install/MONAI/monai/data/wsi_reader.py", line 347, in read
    return self.reader.read(data=data, **kwargs)
  File "/home/dabean/MONAI_install/MONAI/monai/data/wsi_reader.py", line 428, in read
    wsi = CuImage(filename, **kwargs_)
ValueError: Cannot open /PandaChallenge2020/train_images/c2b4cc586b3984b248e7d095d4c0bd03.tiff!

Traceback (most recent call last):
  File "/home/dabean/miniconda3/envs/MONAI_d01_env/lib/python3.9/site-packages/nibabel/loadsave.py", line 90, in load
    stat_result = os.stat(filename)
FileNotFoundError: [Errno 2] No such file or directory: '/PandaChallenge2020/train_images/c2b4cc586b3984b248e7d095d4c0bd03.tiff'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/dabean/MONAI_install/MONAI/monai/transforms/io/array.py", line 230, in __call__
    img = reader.read(filename)
  File "/home/dabean/MONAI_install/MONAI/monai/data/image_reader.py", line 912, in read
    img = nib.load(name, **kwargs_)
  File "/home/dabean/miniconda3/envs/MONAI_d01_env/lib/python3.9/site-packages/nibabel/loadsave.py", line 92, in load
    raise FileNotFoundError(f"No such file or no access: '{filename}'")
FileNotFoundError: No such file or no access: '/PandaChallenge2020/train_images/c2b4cc586b3984b248e7d095d4c0bd03.tiff'

Traceback (most recent call last):
  File "/home/dabean/MONAI_install/MONAI/monai/transforms/io/array.py", line 230, in __call__
    img = reader.read(filename)
  File "/home/dabean/MONAI_install/MONAI/monai/data/image_reader.py", line 1167, in read
    img = PILImage.open(name, **kwargs_)
TypeError: open() got an unexpected keyword argument 'backend'

Traceback (most recent call last):
  File "/home/dabean/MONAI_install/MONAI/monai/transforms/io/array.py", line 230, in __call__
    img = reader.read(filename)
  File "/home/dabean/MONAI_install/MONAI/monai/data/image_reader.py", line 1074, in read
    img = np.load(name, allow_pickle=True, **kwargs_)
TypeError: load() got an unexpected keyword argument 'backend'

Traceback (most recent call last):
  File "/home/dabean/MONAI_install/MONAI/monai/transforms/io/array.py", line 230, in __call__
    img = reader.read(filename)
  File "/home/dabean/MONAI_install/MONAI/monai/data/image_reader.py", line 1548, in read
    nrrd_image = NrrdImage(*nrrd.read(name, index_order=self.index_order, *kwargs_))
TypeError: read() got multiple values for argument 'index_order'

Traceback (most recent call last):
  File "/home/dabean/MONAI_install/MONAI/monai/transforms/io/array.py", line 230, in __call__
    img = reader.read(filename)
  File "/home/dabean/MONAI_install/MONAI/monai/data/image_reader.py", line 267, in read
    img_.append(itk.imread(name, **kwargs_))
TypeError: imread() got an unexpected keyword argument 'backend'

Traceback (most recent call last):
  File "/home/dabean/MONAI_install/MONAI/monai/transforms/io/array.py", line 230, in __call__
    img = reader.read(filename)
  File "/home/dabean/MONAI_install/MONAI/monai/data/image_reader.py", line 481, in read
    ds = pydicom.dcmread(fp=name, **kwargs_)
TypeError: dcmread() got an unexpected keyword argument 'backend'


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/dabean/MONAI_install/MONAI/monai/transforms/transform.py", line 90, in apply_transform
    return _apply_transform(transform, data, unpack_items)
  File "/home/dabean/MONAI_install/MONAI/monai/transforms/transform.py", line 54, in _apply_transform
    return transform(parameters)
  File "/home/dabean/MONAI_install/MONAI/monai/transforms/compose.py", line 173, in __call__
    input_ = apply_transform(_transform, input_, self.map_items, self.unpack_items, self.log_stats)
  File "/home/dabean/MONAI_install/MONAI/monai/transforms/transform.py", line 114, in apply_transform
    raise RuntimeError(f"applying transform {transform}") from e
RuntimeError: applying transform <monai.transforms.io.dictionary.LoadImaged object at 0x7f0d64a66e80>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/dabean/miniconda3/envs/MONAI_d01_env/lib/python3.9/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/dabean/miniconda3/envs/MONAI_d01_env/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/dabean/miniconda3/envs/MONAI_d01_env/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 49, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/dabean/MONAI_install/MONAI/monai/data/dataset.py", line 105, in __getitem__
    return self._transform(index)
  File "/home/dabean/MONAI_install/MONAI/monai/data/dataset.py", line 91, in _transform
    return apply_transform(self.transform, data_i) if self.transform is not None else data_i
  File "/home/dabean/MONAI_install/MONAI/monai/transforms/transform.py", line 114, in apply_transform
    raise RuntimeError(f"applying transform {transform}") from e
RuntimeError: applying transform <monai.transforms.compose.Compose object at 0x7f0d64469bb0>

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions