-
Notifications
You must be signed in to change notification settings - Fork 268
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add CIFAR10 dataset #2
Conversation
Tests are just failing right now because the data is missing on Travis. I think we can add that as with MNIST (we can store it in the cache in Travis's new container environment so that it doesn't need to be redownloaded each time). I'll make a ticket though, because I feel that there is a lot of code duplication between this and MNIST, and there's a variety of things that could be factored out for use by future datasets as well. |
@bartvm Even though I think I've properly added the CIFAR10 files to cache, the tests fail. I'm not very Travis-savvy, is there something I left out? |
On my phone, but the log says: tar: Old option Maybe there's something wrong with the Tar command so that it doesn't actually unpack? |
1 similar comment
On my phone, but the log says: tar: Old option Maybe there's something wrong with the Tar command so that it doesn't actually unpack? |
I think I found what the issue was, but I can't fix it since I don't think I can clear the cache by myself. |
I think the bash command still doesn't fully work. The problem is that curl http://www.cs.utoronto.ca/~kriz/cifar-10-python.tar.gz | tar xzf - |
Yes, I think you're right. I gleaned the command from the Pylearn2 script, but I missed the dash at the end of the
I'll try to push a commit to force resetting the cache properly, and if it works I'll squash all commits related to fixing my mistake. |
@bartvm I wasn't able to repair my cache mistake. I put the right Sorry about that! |
No problem! I cleared the cache manually and restarted the build |
Python 2 passes now, but you should use |
@bartvm Yes! For |
Try opening it with mode |
Sorry, my bad, I only just realised you were trying to unpickle the file that you downloaded, which I guess was pickled with Python 2... They use different encoding schemes, but this could work: cPickle.load(f, encoding='latin1') |
That will fail under Python 2, since |
One option would be this: try:
data = cPickle.load(f, encoding='latin1')
except TypeError:
data = cPickle.load(f) but it isn't very readable in my opinion. |
I agree, but you can use |
@bartvm Thanks for the tip. The tests now pass. |
Fix failing test_prepare_hdf5_file test
Balanced sampling scheme
Based off
MNIST
, uses data that can currently be obtained through Pylearn2'sdownload_cifar10.sh
script.