Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support multi-label classfication #11

Closed
faroit opened this issue Dec 13, 2020 · 0 comments
Closed

Support multi-label classfication #11

faroit opened this issue Dec 13, 2020 · 0 comments
Assignees
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@faroit
Copy link
Contributor

faroit commented Dec 13, 2020

Both, the dataset and the downloader are designed for multi-class classfication tasks. That means that a single label is used to store the data in hierachical folder structure as used in torchvision.datasets.ImageFolder.

Give that we also want to support #9, it can makes sense to switch to the webdataset format where each file is accompanied by a single label (e.g. json) file: resulting in a flat folder without any hierarchy.

eg.

e39871fd9fd74f55.jpg
e39871fd9fd74f55.json
f18b91585c4d3f3e.jpg
f18b91585c4d3f3e.json
ede6e66b2fb59aab.jpg
ede6e66b2fb59aab.json

That would allow us to save all gbif metadata in that json which, in turn, enables more diverse tasks such as unsupervised learning.

Of course we can also off support for both?

@faroit faroit added enhancement New feature or request help wanted Extra attention is needed labels Dec 13, 2020
@faroit faroit self-assigned this Jan 8, 2021
faroit added a commit that referenced this issue Jan 13, 2021
* clean up io, introduce utils

* add support for multi-label classifcation datasets. rename data generator,
@faroit faroit closed this as completed Jan 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant