New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CORe50 Dataset + reporthook for download in Utils #340

Open
wants to merge 6 commits into
base: master
from

Conversation

Projects
None yet
2 participants
@vlomonaco

vlomonaco commented Nov 23, 2017

Hi! This PR would add support to the CORe50 dataset, recently published @ CoRL2017.

I've also added a simple report hook function (credit to Shichao) in the Utils.py file for showing up the status of the download which can be useful for big Datasets.

Note: This implementation it's not super efficient since similar to the ImageFolder class but good for small RAM devices. I plan to update the code with a more efficient pre-loading strategy in the near future.

@alykhantejani

Hi @vlomonaco,

I haven't had time to check the PR fully yet, but instead of adding a report hook function to utils why not use tqdm (if available)?

i.e.

try:
    from tqdm import tqdm
except ImportError:
    tqdm = lambda x:x

Then you can just wrap the loop in tqdm

You can also add something to the README/docstring about optional packages for torchvision (to enhance usability but not required to use it)

@vlomonaco

This comment has been minimized.

Show comment
Hide comment
@vlomonaco

vlomonaco Dec 1, 2017

Hi @alykhantejani! Thank you for the feedback!

Maybe I misunderstood your suggestion but using tqdm with urllib is not that straightforward. You still have to create a report hook function since there's no external loop in a urllib request (see an example here).

Moreover, with just a 10-lines function we can provide download details also to users who don't have tqdm installed. Your call!

vlomonaco commented Dec 1, 2017

Hi @alykhantejani! Thank you for the feedback!

Maybe I misunderstood your suggestion but using tqdm with urllib is not that straightforward. You still have to create a report hook function since there's no external loop in a urllib request (see an example here).

Moreover, with just a 10-lines function we can provide download details also to users who don't have tqdm installed. Your call!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment