[Bug] digits_dann_lightn example command from readme returns an error as it tries to download data from a server.

[bobturneruk]

🐛 Bug

digits_dann_lightn example command from readme returns an error as it tries to download data from a server.

To reproduce

python main.py --cfg configs/MN2UP-CDAN.yaml --gpus 0

** Stack trace/error message **

DAN:
  METHOD: CDAN
  RANDOM_DIM: 1024
  USERANDOM: False
DATASET:
  DIMENSION: 784
  NAME: digits
  NUM_CLASSES: 10
  NUM_REPEAT: 10
  ROOT: ../data
  SIZE_TYPE: source
  SOURCE: mnist
  TARGET: usps
  WEIGHT_TYPE: natural
OUTPUT:
  DIR: ./outputs\digits_mnist2usps
  PB_FRESH: 0
  ROOT: ./outputs
  VERBOSE: False
SOLVER:
  AD_LAMBDA: True
  AD_LR: True
  BASE_LR: 0.001
  INIT_LAMBDA: 1
  MAX_EPOCHS: 120
  MIN_EPOCHS: 20
  MOMENTUM: 0.9
  NESTEROV: True
  SEED: 2020
  TEST_BATCH_SIZE: 200
  TRAIN_BATCH_SIZE: 150
  TYPE: SGD
  WEIGHT_DECAY: 0.0005
==> Building model for seed 2020 ......
Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to ../data\MNIST\raw\train-images-idx3-ubyte.gz
0it [00:00, ?it/s]Traceback (most recent call last):
  File "main.py", line 91, in <module>
    main()
  File "main.py", line 59, in main
    model, train_params = get_model(cfg, dataset, num_channels)
  File "C:\Users\bobturner\Documents\pykale\examples\digits_dann_lightn\model.py", line 98, in get_model
    model = domain_adapter.create_dann_like(
  File "c:\users\bobturner\documents\pykale\kale\pipeline\domain_adapter.py", line 149, in create_dann_like
    return CDANtrainer(
  File "c:\users\bobturner\documents\pykale\kale\pipeline\domain_adapter.py", line 602, in __init__
    super().__init__(dataset, feature_extractor, task_classifier, critic, **base_params)
  File "c:\users\bobturner\documents\pykale\kale\pipeline\domain_adapter.py", line 468, in __init__
    super().__init__(dataset, feature_extractor, task_classifier, **base_params)
  File "c:\users\bobturner\documents\pykale\kale\pipeline\domain_adapter.py", line 276, in __init__
    self._dataset.prepare_data_loaders()
  File "c:\users\bobturner\documents\pykale\kale\loaddata\multi_domain.py", line 132, in prepare_data_loaders
    (self._source_by_split["train"], self._source_by_split["valid"],) = self._source_access.get_train_val(
  File "c:\users\bobturner\documents\pykale\kale\loaddata\dataset_access.py", line 41, in get_train_val
    train_dataset = self.get_train()
  File "c:\users\bobturner\documents\pykale\kale\loaddata\digits_access.py", line 87, in get_train
    return MNIST(self._data_path, train=True, transform=self._transform, download=True)
  File "C:\Users\bobturner\.conda\envs\pykale\lib\site-packages\torchvision\datasets\mnist.py", line 79, in __init__
    self.download()
  File "C:\Users\bobturner\.conda\envs\pykale\lib\site-packages\torchvision\datasets\mnist.py", line 146, in download
    download_and_extract_archive(url, download_root=self.raw_folder, filename=filename, md5=md5)
  File "C:\Users\bobturner\.conda\envs\pykale\lib\site-packages\torchvision\datasets\utils.py", line 256, in download_and_extract_archive
    download_url(url, download_root, filename, md5)
  File "C:\Users\bobturner\.conda\envs\pykale\lib\site-packages\torchvision\datasets\utils.py", line 84, in download_url
    raise e
  File "C:\Users\bobturner\.conda\envs\pykale\lib\site-packages\torchvision\datasets\utils.py", line 70, in download_url
    urllib.request.urlretrieve(
  File "C:\Users\bobturner\.conda\envs\pykale\lib\urllib\request.py", line 247, in urlretrieve
    with contextlib.closing(urlopen(url, data)) as fp:
  File "C:\Users\bobturner\.conda\envs\pykale\lib\urllib\request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Users\bobturner\.conda\envs\pykale\lib\urllib\request.py", line 531, in open
    response = meth(req, response)
  File "C:\Users\bobturner\.conda\envs\pykale\lib\urllib\request.py", line 640, in http_response
    response = self.parent.error(
  File "C:\Users\bobturner\.conda\envs\pykale\lib\urllib\request.py", line 569, in error
    return self._call_chain(*args)
  File "C:\Users\bobturner\.conda\envs\pykale\lib\urllib\request.py", line 502, in _call_chain
    result = func(*args)
  File "C:\Users\bobturner\.conda\envs\pykale\lib\urllib\request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 503: Service Unavailable
0it [00:00, ?it/s]

Expected Behaviour

Code should run without error.

Environment

PyTorch version: 1.7.1+cpu
Is debug build: False
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 10 Pro
GCC version: (tdm64-1) 9.2.0
Clang version: Could not collect
CMake version: Could not collect

Python version: 3.8 (64-bit runtime)
Is CUDA available: False
CUDA runtime version: 10.0.130
GPU models and configuration: GPU 0: GeForce GTX 1050 Ti with Max-Q Design
Nvidia driver version: 452.41
cuDNN version: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\bin\cudnn64_7.dll
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.20.2
[pip3] pytorch-lightning==1.2.10
[pip3] pytorch-memlab==0.2.3
[pip3] torch==1.7.1+cpu
[pip3] torch-cluster==1.5.9
[pip3] torch-geometric==1.7.0
[pip3] torch-scatter==2.0.6
[pip3] torch-sparse==0.6.9
[pip3] torch-spline-conv==1.2.1
[pip3] torchmetrics==0.2.0
[pip3] torchsummary==1.5.1
[pip3] torchvision==0.8.2+cpu
[conda] libblas                   3.9.0                     8_mkl    conda-forge
[conda] libcblas                  3.9.0                     8_mkl    conda-forge
[conda] liblapack                 3.9.0                     8_mkl    conda-forge
[conda] mkl                       2020.4             hb70f87d_311    conda-forge
[conda] numpy                     1.20.2           py38h09042cb_0    conda-forge
[conda] pytorch-lightning         1.2.10                   pypi_0    pypi
[conda] pytorch-memlab            0.2.3                    pypi_0    pypi
[conda] torch                     1.7.1+cpu                pypi_0    pypi
[conda] torch-cluster             1.5.9                    pypi_0    pypi
[conda] torch-geometric           1.7.0                    pypi_0    pypi
[conda] torch-scatter             2.0.6                    pypi_0    pypi
[conda] torch-sparse              0.6.9                    pypi_0    pypi
[conda] torch-spline-conv         1.2.1                    pypi_0    pypi
[conda] torchmetrics              0.2.0                    pypi_0    pypi
[conda] torchsummary              1.5.1                    pypi_0    pypi
[conda] torchvision               0.8.2+cpu                pypi_0    pypi

[bobturneruk]

Server response in browser:

[bobturneruk]

Is there a more reliable source for this data, I wonder?

[haipinglu]

Is there a more reliable source for this data, I wonder?

Even if there are, the MNIST is using pytorch's API and changes in pytorch are needed to fix the issue, unless we reimplement the MNIST API.

There are four digits datasets. My suggestion is to avoid MNIST if it is not stable and use the other three (change the config). PyKale has APIs for UPSP and MNISTM, i.e. we can modify so you could use these two. SVHN is also stable. We can talk about this today. Thanks.

[bobturneruk]

Sure I note both the examples in the readme have this:

python main.py --cfg configs/MN2UP-CDAN.yaml --gpus 0

python main.py --cfg configs/MN2UP-DANN.yaml --gpus 0

Maybe I can help change.

[bobturneruk]

Will fix by changing data sources in examples - other are available.