Skip to content

add work around for pytorch downloading mnist data #315

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 3, 2021
Merged

add work around for pytorch downloading mnist data #315

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2e849e9
add work around for pytorch downloading mnist data, found in later se…
mitchjablonski Apr 9, 2021
51cc7b8
updates to support downloading mnist data in autoencoder section
mitchjablonski Apr 9, 2021
1ebb724
adds the note about pytorch issue, and moves the download directory t…
mitchjablonski Apr 9, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 19 additions & 4 deletions autoencoder/convolutional-autoencoder/Convolutional_Autoencoder_Exercise.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,21 @@
"Let's get started by importing our libraries and getting the dataset."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# The MNIST datasets are hosted on yann.lecun.com that has moved under CloudFlare protection\n",
"# Run this script to enable the datasets download\n",
"# Reference: https://github.com/pytorch/vision/issues/1938\n",
"from six.moves import urllib\n",
"opener = urllib.request.build_opener()\n",
"opener.addheaders = [('User-agent', 'Mozilla/5.0')]\n",
"urllib.request.install_opener(opener)"
]
},
{
"cell_type": "code",
"execution_count": null,
Expand All @@ -38,9 +53,9 @@
"transform = transforms.ToTensor()\n",
"\n",
"# load the training and test datasets\n",
"train_data = datasets.MNIST(root='data', train=True,\n",
"train_data = datasets.MNIST(root='~/.pytorch/MNIST_data/', train=True,\n",
" download=True, transform=transform)\n",
"test_data = datasets.MNIST(root='data', train=False,\n",
"test_data = datasets.MNIST(root='~/.pytorch/MNIST_data/', train=False,\n",
" download=True, transform=transform)"
]
},
Expand Down Expand Up @@ -282,7 +297,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python [default]",
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
Expand All @@ -296,7 +311,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.4"
"version": "3.7.1"
}
},
"nbformat": 4,
Expand Down
23 changes: 19 additions & 4 deletions autoencoder/convolutional-autoencoder/Convolutional_Autoencoder_Solution.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,21 @@
"Let's get started by importing our libraries and getting the dataset."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# The MNIST datasets are hosted on yann.lecun.com that has moved under CloudFlare protection\n",
"# Run this script to enable the datasets download\n",
"# Reference: https://github.com/pytorch/vision/issues/1938\n",
"from six.moves import urllib\n",
"opener = urllib.request.build_opener()\n",
"opener.addheaders = [('User-agent', 'Mozilla/5.0')]\n",
"urllib.request.install_opener(opener)"
]
},
{
"cell_type": "code",
"execution_count": 1,
Expand All @@ -38,9 +53,9 @@
"transform = transforms.ToTensor()\n",
"\n",
"# load the training and test datasets\n",
"train_data = datasets.MNIST(root='data', train=True,\n",
"train_data = datasets.MNIST(root='~/.pytorch/MNIST_data/', train=True,\n",
" download=True, transform=transform)\n",
"test_data = datasets.MNIST(root='data', train=False,\n",
"test_data = datasets.MNIST(root='~/.pytorch/MNIST_data/', train=False,\n",
" download=True, transform=transform)"
]
},
Expand Down Expand Up @@ -373,7 +388,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python [default]",
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
Expand All @@ -387,7 +402,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.4"
"version": "3.7.1"
}
},
"nbformat": 4,
Expand Down
23 changes: 19 additions & 4 deletions autoencoder/convolutional-autoencoder/Upsampling_Solution.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,21 @@
"Let's get started by importing our libraries and getting the dataset."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# The MNIST datasets are hosted on yann.lecun.com that has moved under CloudFlare protection\n",
"# Run this script to enable the datasets download\n",
"# Reference: https://github.com/pytorch/vision/issues/1938\n",
"from six.moves import urllib\n",
"opener = urllib.request.build_opener()\n",
"opener.addheaders = [('User-agent', 'Mozilla/5.0')]\n",
"urllib.request.install_opener(opener)"
]
},
{
"cell_type": "code",
"execution_count": 1,
Expand All @@ -38,9 +53,9 @@
"transform = transforms.ToTensor()\n",
"\n",
"# load the training and test datasets\n",
"train_data = datasets.MNIST(root='data', train=True,\n",
"train_data = datasets.MNIST(root='~/.pytorch/MNIST_data/', train=True,\n",
" download=True, transform=transform)\n",
"test_data = datasets.MNIST(root='data', train=False,\n",
"test_data = datasets.MNIST(root='~/.pytorch/MNIST_data/', train=False,\n",
" download=True, transform=transform)"
]
},
Expand Down Expand Up @@ -365,7 +380,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python [default]",
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
Expand All @@ -379,7 +394,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.4"
"version": "3.7.1"
}
},
"nbformat": 4,
Expand Down
23 changes: 19 additions & 4 deletions autoencoder/denoising-autoencoder/Denoising_Autoencoder_Exercise.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,21 @@
"Let's get started by importing our libraries and getting the dataset."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# The MNIST datasets are hosted on yann.lecun.com that has moved under CloudFlare protection\n",
"# Run this script to enable the datasets download\n",
"# Reference: https://github.com/pytorch/vision/issues/1938\n",
"from six.moves import urllib\n",
"opener = urllib.request.build_opener()\n",
"opener.addheaders = [('User-agent', 'Mozilla/5.0')]\n",
"urllib.request.install_opener(opener)"
]
},
{
"cell_type": "code",
"execution_count": null,
Expand All @@ -30,9 +45,9 @@
"transform = transforms.ToTensor()\n",
"\n",
"# load the training and test datasets\n",
"train_data = datasets.MNIST(root='data', train=True,\n",
"train_data = datasets.MNIST(root='~/.pytorch/MNIST_data/', train=True,\n",
" download=True, transform=transform)\n",
"test_data = datasets.MNIST(root='data', train=False,\n",
"test_data = datasets.MNIST(root='~/.pytorch/MNIST_data/', train=False,\n",
" download=True, transform=transform)\n",
"\n",
"# Create training and test dataloaders\n",
Expand Down Expand Up @@ -273,7 +288,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python [default]",
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
Expand All @@ -287,7 +302,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.4"
"version": "3.7.1"
}
},
"nbformat": 4,
Expand Down
23 changes: 19 additions & 4 deletions autoencoder/denoising-autoencoder/Denoising_Autoencoder_Solution.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,21 @@
"Let's get started by importing our libraries and getting the dataset."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# The MNIST datasets are hosted on yann.lecun.com that has moved under CloudFlare protection\n",
"# Run this script to enable the datasets download\n",
"# Reference: https://github.com/pytorch/vision/issues/1938\n",
"from six.moves import urllib\n",
"opener = urllib.request.build_opener()\n",
"opener.addheaders = [('User-agent', 'Mozilla/5.0')]\n",
"urllib.request.install_opener(opener)"
]
},
{
"cell_type": "code",
"execution_count": 1,
Expand All @@ -28,9 +43,9 @@
"transform = transforms.ToTensor()\n",
"\n",
"# load the training and test datasets\n",
"train_data = datasets.MNIST(root='data', train=True,\n",
"train_data = datasets.MNIST(root='~/.pytorch/MNIST_data/', train=True,\n",
" download=True, transform=transform)\n",
"test_data = datasets.MNIST(root='data', train=False,\n",
"test_data = datasets.MNIST(root='~/.pytorch/MNIST_data/', train=False,\n",
" download=True, transform=transform)\n",
"\n",
"# Create training and test dataloaders\n",
Expand Down Expand Up @@ -369,7 +384,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python [default]",
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
Expand All @@ -383,7 +398,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.4"
"version": "3.7.1"
}
},
"nbformat": 4,
Expand Down
27 changes: 20 additions & 7 deletions autoencoder/linear-autoencoder/Simple_Autoencoder_Exercise.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,21 @@
"In this notebook, we'll be build a simple network architecture for the encoder and decoder. Let's get started by importing our libraries and getting the dataset."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# The MNIST datasets are hosted on yann.lecun.com that has moved under CloudFlare protection\n",
"# Run this script to enable the datasets download\n",
"# Reference: https://github.com/pytorch/vision/issues/1938\n",
"from six.moves import urllib\n",
"opener = urllib.request.build_opener()\n",
"opener.addheaders = [('User-agent', 'Mozilla/5.0')]\n",
"urllib.request.install_opener(opener)"
]
},
{
"cell_type": "code",
"execution_count": null,
Expand All @@ -36,9 +51,9 @@
"transform = transforms.ToTensor()\n",
"\n",
"# load the training and test datasets\n",
"train_data = datasets.MNIST(root='data', train=True,\n",
"train_data = datasets.MNIST(root='~/.pytorch/MNIST_data/', train=True,\n",
" download=True, transform=transform)\n",
"test_data = datasets.MNIST(root='data', train=False,\n",
"test_data = datasets.MNIST(root='~/.pytorch/MNIST_data/', train=False,\n",
" download=True, transform=transform)"
]
},
Expand Down Expand Up @@ -267,9 +282,7 @@
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"metadata": {},
"source": [
"## Up Next\n",
"\n",
Expand All @@ -279,7 +292,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python [default]",
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
Expand All @@ -293,7 +306,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.4"
"version": "3.7.1"
}
},
"nbformat": 4,
Expand Down
27 changes: 20 additions & 7 deletions autoencoder/linear-autoencoder/Simple_Autoencoder_Solution.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,21 @@
"In this notebook, we'll be build a simple network architecture for the encoder and decoder. Let's get started by importing our libraries and getting the dataset."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# The MNIST datasets are hosted on yann.lecun.com that has moved under CloudFlare protection\n",
"# Run this script to enable the datasets download\n",
"# Reference: https://github.com/pytorch/vision/issues/1938\n",
"from six.moves import urllib\n",
"opener = urllib.request.build_opener()\n",
"opener.addheaders = [('User-agent', 'Mozilla/5.0')]\n",
"urllib.request.install_opener(opener)"
]
},
{
"cell_type": "code",
"execution_count": 1,
Expand All @@ -36,9 +51,9 @@
"transform = transforms.ToTensor()\n",
"\n",
"# load the training and test datasets\n",
"train_data = datasets.MNIST(root='data', train=True,\n",
"train_data = datasets.MNIST(root='~/.pytorch/MNIST_data/', train=True,\n",
" download=True, transform=transform)\n",
"test_data = datasets.MNIST(root='data', train=False,\n",
"test_data = datasets.MNIST(root='~/.pytorch/MNIST_data/', train=False,\n",
" download=True, transform=transform)"
]
},
Expand Down Expand Up @@ -336,9 +351,7 @@
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"metadata": {},
"source": [
"## Up Next\n",
"\n",
Expand All @@ -348,7 +361,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python [default]",
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
Expand All @@ -362,7 +375,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.4"
"version": "3.7.1"
}
},
"nbformat": 4,
Expand Down
Loading