test error 2d_classification/mednist_tutorial.ipynb #797
Description
Describe the bug
10:08:02 Running ./2d_classification/mednist_tutorial.ipynb
10:08:02 Checking PEP8 compliance...
10:08:03 Running notebook...
10:08:03 Before:
10:08:03 "max_epochs = 4\n",
10:08:03 After:
10:08:03 "max_epochs = 1\n",
10:08:03 Before:
10:08:03 "val_interval = 1\n",
10:08:03 After:
10:08:03 "val_interval = 1\n",
10:08:07 MONAI version: 0.9.1rc3+3.gec4c0311
10:08:07 Numpy version: 1.22.4
10:08:07 Pytorch version: 1.10.2+cu102
10:08:07 MONAI flags: HAS_EXT = False, USE_COMPILED = False, USE_META_DICT = False
10:08:07 MONAI rev id: ec4c03115ba45b2e950103a24db22f16d54c8d40
10:08:07 MONAI __file__: /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/__init__.py
10:08:07
10:08:07 Optional dependencies:
10:08:07 Pytorch Ignite version: 0.4.9
10:08:07 Nibabel version: 4.0.1
10:08:07 scikit-image version: 0.19.3
10:08:07 Pillow version: 7.0.0
10:08:07 Tensorboard version: 2.9.1
10:08:07 gdown version: 4.5.1
10:08:07 TorchVision version: 0.11.3+cu102
10:08:07 tqdm version: 4.64.0
10:08:07 lmdb version: 1.3.0
10:08:07 psutil version: 5.9.1
10:08:07 pandas version: 1.1.5
10:08:07 einops version: 0.4.1
10:08:07 transformers version: 4.20.1
10:08:07 mlflow version: 1.27.0
10:08:07 pynrrd version: 0.4.3
10:08:07
10:08:07 For details about installing the optional dependencies, please visit:
10:08:07 https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies
10:08:07
10:08:07 /opt/conda/lib/python3.8/site-packages/papermill/iorw.py:58: FutureWarning: pyarrow.HadoopFileSystem is deprecated as of 2.0.0, please use pyarrow.fs.HadoopFileSystem instead.
10:08:07 from pyarrow import HadoopFileSystem
10:08:34
Executing: 0%| | 0/31 [00:00<?, ?cell/s]
Executing: 3%|▎ | 1/31 [00:01<00:44, 1.49s/cell]
Executing: 10%|▉ | 3/31 [00:06<01:04, 2.31s/cell]
Executing: 16%|█▌ | 5/31 [00:09<00:45, 1.74s/cell]
Executing: 29%|██▉ | 9/31 [00:17<00:44, 2.02s/cell]
Executing: 42%|████▏ | 13/31 [00:18<00:19, 1.11s/cell]
Executing: 48%|████▊ | 15/31 [00:18<00:15, 1.06cell/s]
Executing: 71%|███████ | 22/31 [00:22<00:06, 1.43cell/s]
Executing: 77%|███████▋ | 24/31 [00:23<00:04, 1.42cell/s]
Executing: 77%|███████▋ | 24/31 [00:26<00:07, 1.12s/cell]
10:08:34 Traceback (most recent call last):
10:08:34 File "/opt/conda/bin/papermill", line 8, in <module>
10:08:34 sys.exit(papermill())
10:08:34 File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 1128, in __call__
10:08:34 return self.main(*args, **kwargs)
10:08:34 File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 1053, in main
10:08:34 rv = self.invoke(ctx)
10:08:34 File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
10:08:34 return ctx.invoke(self.callback, **ctx.params)
10:08:34 File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 754, in invoke
10:08:34 return __callback(*args, **kwargs)
10:08:34 File "/opt/conda/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
10:08:34 return f(get_current_context(), *args, **kwargs)
10:08:34 File "/opt/conda/lib/python3.8/site-packages/papermill/cli.py", line 250, in papermill
10:08:34 execute_notebook(
10:08:34 File "/opt/conda/lib/python3.8/site-packages/papermill/execute.py", line 122, in execute_notebook
10:08:34 raise_for_execution_errors(nb, output_path)
10:08:34 File "/opt/conda/lib/python3.8/site-packages/papermill/execute.py", line 234, in raise_for_execution_errors
10:08:34 raise error
10:08:34 papermill.exceptions.PapermillExecutionError:
10:08:34 ---------------------------------------------------------------------------
10:08:34 Exception encountered at "In [12]":
10:08:34 ---------------------------------------------------------------------------
10:08:34 RuntimeError Traceback (most recent call last)
10:08:34 Input In [12], in <cell line: 6>()
10:08:34 10 epoch_loss = 0
10:08:34 11 step = 0
10:08:34 ---> 12 for batch_data in train_loader:
10:08:34 13 step += 1
10:08:34 14 inputs, labels = batch_data[0].to(device), batch_data[1].to(device)
10:08:34
10:08:34 File /opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py:521, in _BaseDataLoaderIter.__next__(self)
10:08:34 519 if self._sampler_iter is None:
10:08:34 520 self._reset()
10:08:34 --> 521 data = self._next_data()
10:08:34 522 self._num_yielded += 1
10:08:34 523 if self._dataset_kind == _DatasetKind.Iterable and \
10:08:34 524 self._IterableDataset_len_called is not None and \
10:08:34 525 self._num_yielded > self._IterableDataset_len_called:
10:08:34
10:08:34 File /opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py:1203, in _MultiProcessingDataLoaderIter._next_data(self)
10:08:34 1201 else:
10:08:34 1202 del self._task_info[idx]
10:08:34 -> 1203 return self._process_data(data)
10:08:34
10:08:34 File /opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py:1229, in _MultiProcessingDataLoaderIter._process_data(self, data)
10:08:34 1227 self._try_put_index()
10:08:34 1228 if isinstance(data, ExceptionWrapper):
10:08:34 -> 1229 data.reraise()
10:08:34 1230 return data
10:08:34
10:08:34 File /opt/conda/lib/python3.8/site-packages/torch/_utils.py:434, in ExceptionWrapper.reraise(self)
10:08:34 430 except TypeError:
10:08:34 431 # If the exception takes multiple arguments, don't try to
10:08:34 432 # instantiate since we don't know how to
10:08:34 433 raise RuntimeError(msg) from None
10:08:34 --> 434 raise exception
10:08:34
10:08:34 RuntimeError: Caught RuntimeError in DataLoader worker process 0.
10:08:34 Original Traceback (most recent call last):
10:08:34 File "/home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/transforms/transform.py", line 89, in apply_transform
10:08:34 return [_apply_transform(transform, item, unpack_items) for item in data]
10:08:34 File "/home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/transforms/transform.py", line 89, in <listcomp>
10:08:34 return [_apply_transform(transform, item, unpack_items) for item in data]
10:08:34 File "/home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/transforms/transform.py", line 54, in _apply_transform
10:08:34 return transform(parameters)
10:08:34 File "/home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/transforms/utility/array.py", line 196, in __call__
10:08:34 out: NdarrayOrTensor = convert_to_tensor(img[None], track_meta=get_track_meta())
10:08:34 KeyError: None
10:08:34
10:08:34 The above exception was the direct cause of the following exception:
10:08:34
10:08:34 Traceback (most recent call last):
10:08:34 File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
10:08:34 data = fetcher.fetch(index)
10:08:34 File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
10:08:34 data = [self.dataset[idx] for idx in possibly_batched_index]
10:08:34 File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 49, in <listcomp>
10:08:34 data = [self.dataset[idx] for idx in possibly_batched_index]
10:08:34 File "/tmp/ipykernel_24084/3806419499.py", line 11, in __getitem__
10:08:34 return self.transforms(self.image_files[index]), self.labels[index]
10:08:34 File "/home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/transforms/compose.py", line 173, in __call__
10:08:34 input_ = apply_transform(_transform, input_, self.map_items, self.unpack_items, self.log_stats)
10:08:34 File "/home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/transforms/transform.py", line 114, in apply_transform
10:08:34 raise RuntimeError(f"applying transform {transform}") from e
10:08:34 RuntimeError: applying transform <monai.transforms.utility.array.AddChannel object at 0x7fb656329d60>