Skip to content

dataset downloading issue #1510

@wyli

Description

@wyli
[2023-09-08T01:43:38.188Z] Running ./modules/transforms_demo_2d.ipynb
[2023-09-08T01:43:38.188Z] Checking PEP8 compliance...
[2023-09-08T01:43:38.752Z] Running notebook...
[2023-09-08T01:43:46.854Z] MONAI version: 1.2.0+110.gef7debe4
[2023-09-08T01:43:46.854Z] Numpy version: 1.26.0rc1
[2023-09-08T01:43:46.854Z] Pytorch version: 2.1.0a0+29c30b1
[2023-09-08T01:43:46.854Z] MONAI flags: HAS_EXT = False, USE_COMPILED = False, USE_META_DICT = False
[2023-09-08T01:43:46.854Z] MONAI rev id: ef7debe47603f882513b255e449b26659766b64f
[2023-09-08T01:43:46.854Z] MONAI __file__: /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/__init__.py
[2023-09-08T01:43:46.854Z] 
[2023-09-08T01:43:46.854Z] Optional dependencies:
[2023-09-08T01:43:46.854Z] Pytorch Ignite version: 0.4.11
[2023-09-08T01:43:46.854Z] ITK version: 5.3.0
[2023-09-08T01:43:46.854Z] Nibabel version: 5.1.0
[2023-09-08T01:43:46.854Z] scikit-image version: 0.21.0
[2023-09-08T01:43:46.854Z] scipy version: 1.11.1
[2023-09-08T01:43:46.854Z] Pillow version: 9.2.0
[2023-09-08T01:43:46.854Z] Tensorboard version: 2.9.0
[2023-09-08T01:43:46.854Z] gdown version: 4.7.1
[2023-09-08T01:43:46.854Z] TorchVision version: 0.16.0a0
[2023-09-08T01:43:46.854Z] tqdm version: 4.65.0
[2023-09-08T01:43:46.854Z] lmdb version: 1.4.1
[2023-09-08T01:43:46.854Z] psutil version: 5.9.4
[2023-09-08T01:43:46.854Z] pandas version: 2.1.0
[2023-09-08T01:43:46.854Z] einops version: 0.6.1
[2023-09-08T01:43:46.854Z] transformers version: 4.21.3
[2023-09-08T01:43:46.854Z] mlflow version: 2.6.0
[2023-09-08T01:43:46.854Z] pynrrd version: 1.0.0
[2023-09-08T01:43:46.854Z] clearml version: 1.12.2
[2023-09-08T01:43:46.854Z] 
[2023-09-08T01:43:46.854Z] For details about installing the optional dependencies, please visit:
[2023-09-08T01:43:46.854Z]     https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies
[2023-09-08T01:43:46.854Z] 
[2023-09-08T01:43:48.745Z] papermill  --progress-bar -k python3
[2023-09-08T01:43:49.003Z] /usr/local/lib/python3.10/dist-packages/papermill/iorw.py:153: UserWarning: the file is not specified with any extension : -
[2023-09-08T01:43:49.004Z]   warnings.warn(
[2023-09-08T01:44:54.678Z] 
Executing:   0%|          | 0/21 [00:00<?, ?cell/s]
Executing:   5%|▍         | 1/21 [00:01<00:29,  1.48s/cell]
Executing:  19%|█▉        | 4/21 [00:10<00:45,  2.68s/cell]
Executing:  29%|██▊       | 6/21 [00:16<00:42,  2.81s/cell]
Executing:  48%|████▊     | 10/21 [01:03<01:25,  7.77s/cell]
Executing:  48%|████▊     | 10/21 [01:05<01:11,  6.52s/cell]
[2023-09-08T01:44:54.678Z] /usr/local/lib/python3.10/dist-packages/papermill/iorw.py:153: UserWarning: the file is not specified with any extension : -
[2023-09-08T01:44:54.678Z]   warnings.warn(
[2023-09-08T01:44:54.678Z] Traceback (most recent call last):
[2023-09-08T01:44:54.678Z]   File "/usr/local/bin/papermill", line 8, in <module>
[2023-09-08T01:44:54.678Z]     sys.exit(papermill())
[2023-09-08T01:44:54.678Z]   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1157, in __call__
[2023-09-08T01:44:54.678Z]     return self.main(*args, **kwargs)
[2023-09-08T01:44:54.678Z]   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1078, in main
[2023-09-08T01:44:54.678Z]     rv = self.invoke(ctx)
[2023-09-08T01:44:54.678Z]   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1434, in invoke
[2023-09-08T01:44:54.678Z]     return ctx.invoke(self.callback, **ctx.params)
[2023-09-08T01:44:54.678Z]   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 783, in invoke
[2023-09-08T01:44:54.678Z]     return __callback(*args, **kwargs)
[2023-09-08T01:44:54.678Z]   File "/usr/local/lib/python3.10/dist-packages/click/decorators.py", line 33, in new_func
[2023-09-08T01:44:54.678Z]     return f(get_current_context(), *args, **kwargs)
[2023-09-08T01:44:54.678Z]   File "/usr/local/lib/python3.10/dist-packages/papermill/cli.py", line 250, in papermill
[2023-09-08T01:44:54.678Z]     execute_notebook(
[2023-09-08T01:44:54.678Z]   File "/usr/local/lib/python3.10/dist-packages/papermill/execute.py", line 128, in execute_notebook
[2023-09-08T01:44:54.678Z]     raise_for_execution_errors(nb, output_path)
[2023-09-08T01:44:54.678Z]   File "/usr/local/lib/python3.10/dist-packages/papermill/execute.py", line 232, in raise_for_execution_errors
[2023-09-08T01:44:54.678Z]     raise error
[2023-09-08T01:44:54.678Z] papermill.exceptions.PapermillExecutionError: 
[2023-09-08T01:44:54.678Z] ---------------------------------------------------------------------------
[2023-09-08T01:44:54.678Z] Exception encountered at "In [4]":
[2023-09-08T01:44:54.678Z] ---------------------------------------------------------------------------
[2023-09-08T01:44:54.678Z] ContentTooShortError                      Traceback (most recent call last)
[2023-09-08T01:44:54.678Z] Cell In[4], line 10
[2023-09-08T01:44:54.678Z]       8 data_dir = os.path.join(root_dir, "Warwick QU Dataset (Released 2016_07_08)")
[2023-09-08T01:44:54.678Z]       9 if not os.path.exists(data_dir):
[2023-09-08T01:44:54.678Z] ---> 10     download_and_extract(resource, compressed_file, root_dir, md5)
[2023-09-08T01:44:54.678Z] 
[2023-09-08T01:44:54.678Z] File /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/apps/utils.py:328, in download_and_extract(url, filepath, output_dir, hash_val, hash_type, file_type, has_base, progress)
[2023-09-08T01:44:54.678Z]     326 with tempfile.TemporaryDirectory() as tmp_dir:
[2023-09-08T01:44:54.678Z]     327     filename = filepath or Path(tmp_dir, _basename(url)).resolve()
[2023-09-08T01:44:54.678Z] --> 328     download_url(url=url, filepath=filename, hash_val=hash_val, hash_type=hash_type, progress=progress)
[2023-09-08T01:44:54.678Z]     329     extractall(filepath=filename, output_dir=output_dir, file_type=file_type, has_base=has_base)
[2023-09-08T01:44:54.678Z] 
[2023-09-08T01:44:54.678Z] File /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/apps/utils.py:219, in download_url(url, filepath, hash_val, hash_type, progress, **gdown_kwargs)
[2023-09-08T01:44:54.678Z]     214             raise RuntimeError(
[2023-09-08T01:44:54.678Z]     215                 f"Download of file from {download_url}, received from {url} "
[2023-09-08T01:44:54.678Z]     216                 + f" to {filepath} failed due to network issue or denied permission."
[2023-09-08T01:44:54.678Z]     217             )
[2023-09-08T01:44:54.678Z]     218 else:
[2023-09-08T01:44:54.678Z] --> 219     _download_with_progress(url, tmp_name, progress=progress)
[2023-09-08T01:44:54.678Z]     220 if not tmp_name.exists():
[2023-09-08T01:44:54.678Z]     221     raise RuntimeError(
[2023-09-08T01:44:54.678Z]     222         f"Download of file from {url} to {filepath} failed due to network issue or denied permission."
[2023-09-08T01:44:54.678Z]     223     )
[2023-09-08T01:44:54.678Z] 
[2023-09-08T01:44:54.678Z] File /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/apps/utils.py:119, in _download_with_progress(url, filepath, progress)
[2023-09-08T01:44:54.678Z]     117 except (URLError, HTTPError, ContentTooShortError, OSError) as e:
[2023-09-08T01:44:54.678Z]     118     logger.error(f"Download failed from {url} to {filepath}.")
[2023-09-08T01:44:54.678Z] --> 119     raise e
[2023-09-08T01:44:54.678Z] 
[2023-09-08T01:44:54.678Z] File /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/apps/utils.py:112, in _download_with_progress(url, filepath, progress)
[2023-09-08T01:44:54.678Z]     109             self.update(b * bsize - self.n)  # will also set self.n = b * bsize
[2023-09-08T01:44:54.678Z]     111     with TqdmUpTo(unit="B", unit_scale=True, unit_divisor=1024, miniters=1, desc=_basename(filepath)) as t:
[2023-09-08T01:44:54.678Z] --> 112         urlretrieve(url, filepath, reporthook=t.update_to)
[2023-09-08T01:44:54.678Z]     113 else:
[2023-09-08T01:44:54.678Z]     114     if not has_tqdm and progress:
[2023-09-08T01:44:54.678Z] 
[2023-09-08T01:44:54.678Z] File /usr/lib/python3.10/urllib/request.py:280, in urlretrieve(url, filename, reporthook, data)
[2023-09-08T01:44:54.678Z]     277                 reporthook(blocknum, bs, size)
[2023-09-08T01:44:54.678Z]     279 if size >= 0 and read < size:
[2023-09-08T01:44:54.678Z] --> 280     raise ContentTooShortError(
[2023-09-08T01:44:54.678Z]     281         "retrieval incomplete: got only %i out of %i bytes"
[2023-09-08T01:44:54.678Z]     282         % (read, size), result)
[2023-09-08T01:44:54.678Z]     284 return result
[2023-09-08T01:44:54.678Z] 
[2023-09-08T01:44:54.678Z] ContentTooShortError: <urlopen error retrieval incomplete: got only 117802284 out of 180902609 bytes>

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions