TestMnist failed with http error

[JackCaoG]

Test currently failed with

Mar 03 23:01:28 + python test/test_train_mnist.py --tidy
0it [00:00, ?it/s]EDownloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to /tmp/mnist-data/MNIST/raw/train-images-idx3-ubyte.gz
Mar 03 23:01:30 
Mar 03 23:01:30 ======================================================================
Mar 03 23:01:30 ERROR: test_accurracy (__main__.TrainMnist)
Mar 03 23:01:30 ----------------------------------------------------------------------
Mar 03 23:01:30 Traceback (most recent call last):
Mar 03 23:01:30   File "test/test_train_mnist.py", line 186, in test_accurracy
Mar 03 23:01:30     self.assertGreaterEqual(train_mnist(), FLAGS.target_accuracy)
Mar 03 23:01:30   File "test/test_train_mnist.py", line 74, in train_mnist
Mar 03 23:01:30     transforms.Normalize((0.1307,), (0.3081,))]))
Mar 03 23:01:30   File "/var/lib/jenkins/.local/lib/python3.6/site-packages/torchvision/datasets/mnist.py", line 79, in __init__
Mar 03 23:01:30     self.download()
Mar 03 23:01:30   File "/var/lib/jenkins/.local/lib/python3.6/site-packages/torchvision/datasets/mnist.py", line 146, in download
Mar 03 23:01:30     download_and_extract_archive(url, download_root=self.raw_folder, filename=filename, md5=md5)
Mar 03 23:01:30   File "/var/lib/jenkins/.local/lib/python3.6/site-packages/torchvision/datasets/utils.py", line 299, in download_and_extract_archive
Mar 03 23:01:30     download_url(url, download_root, filename, md5)
Mar 03 23:01:30   File "/var/lib/jenkins/.local/lib/python3.6/site-packages/torchvision/datasets/utils.py", line 125, in download_url
Mar 03 23:01:30     raise e
Mar 03 23:01:30   File "/var/lib/jenkins/.local/lib/python3.6/site-packages/torchvision/datasets/utils.py", line 113, in download_url
Mar 03 23:01:30     reporthook=gen_bar_updater()
Mar 03 23:01:30   File "/opt/conda/lib/python3.6/urllib/request.py", line 248, in urlretrieve
Mar 03 23:01:30     with contextlib.closing(urlopen(url, data)) as fp:
Mar 03 23:01:30   File "/opt/conda/lib/python3.6/urllib/request.py", line 223, in urlopen
Mar 03 23:01:30     return opener.open(url, data, timeout)
Mar 03 23:01:30   File "/opt/conda/lib/python3.6/urllib/request.py", line 532, in open
Mar 03 23:01:30     response = meth(req, response)
Mar 03 23:01:30   File "/opt/conda/lib/python3.6/urllib/request.py", line 642, in http_response
Mar 03 23:01:30     'http', request, response, code, msg, hdrs)
Mar 03 23:01:30   File "/opt/conda/lib/python3.6/urllib/request.py", line 570, in error
Mar 03 23:01:30     return self._call_chain(*args)
Mar 03 23:01:30   File "/opt/conda/lib/python3.6/urllib/request.py", line 504, in _call_chain
Mar 03 23:01:30     result = func(*args)
Mar 03 23:01:30   File "/opt/conda/lib/python3.6/urllib/request.py", line 650, in http_error_default
Mar 03 23:01:30     raise HTTPError(req.full_url, code, msg, hdrs, fp)
Mar 03 23:01:30 urllib.error.HTTPError: HTTP Error 403: Forbidden
Mar 03 23:01:30 
Mar 03 23:01:30 ----------------------------------------------------------------------
Mar 03 23:01:30 Ran 1 test in 0.541s
Mar 03 23:01:30 
Mar 03 23:01:30 FAILED (errors=1)
0it [00:00, ?it/s]

[zcain117]

Could we just use --fake_data all the time?

[taylanbil]

that would mean we're not testing the convergence. Can we download the data to a PD and use that pd instead of having a dependency to download?

[zcain117]

The PR #2809 was regarding circleCI, idk if they have any native persistent disk options

for xl-ml-test we can definitely use a PD

[taylanbil]

ok then I propose:

• the dependency to download is not ideal. let's get rid of it, use --fake_data
• Make the fake data generator contain some pattern that model could learn. something like, generate the tensor, and multiply it w/ 2 if the target is 1 and w/ 1 if the target is 0.
• This way, we still somewhat test for convergence in circleci.

[cpuhrsch]

This is the approach torchvision took: https://github.com/pytorch/vision/pull/3499/files

[cpuhrsch]

since XLA depends on torchvision for the dataset here, this fix will soon propagate from torchvision

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.