Add ImageFolderDataset
#1232
Add ImageFolderDataset
#1232
Conversation
|
Note that I added an We could remove the options until they are implemented. Let me know your thoughts :) /edit: welp, looks like I have some tests to fix tomorrow. |
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #1232 +/- ##
==========================================
- Coverage 84.41% 84.40% -0.02%
==========================================
Files 549 550 +1
Lines 61952 62174 +222
==========================================
+ Hits 52295 52475 +180
- Misses 9657 9699 +42 ☔ View full report in Codecov by Sentry. |
| /// | ||
| /// Taken from [burn-dataset](https://github.com/tracel-ai/burn/blob/main/burn-dataset/src/vision/downloader.rs). | ||
| #[tokio::main(flavor = "current_thread")] | ||
| pub async fn download_file_as_bytes(url: &str, message: &str) -> Vec<u8> { |
There was a problem hiding this comment.
It seems to be duplicated from the mnist dataset implementation right? Maybe we can actually add cifar to burn-dataset and reuse the setup from mnist?
There was a problem hiding this comment.
Yes, that's why I linked to it. That download function can be used for any other future dataset we might want to add, I just wasn't sure if we wanted to add CIFAR-10 explicitly so I put it as an example. And the downloader module is private so I copied the code over with a reference.
Btw the original CIFAR-10 source is in bytes form (I used a mirror with the folder structure to illustrate the ImageFolderDataset which is more common), closer to MNIST.
| TrainingConfig::new(SgdConfig::new().with_momentum(Some(MomentumConfig { | ||
| momentum: 0.9, | ||
| dampening: 0., | ||
| nesterov: false, | ||
| }))), |
There was a problem hiding this comment.
The question is: is Adam better in this case 😅
There was a problem hiding this comment.
I actually haven't fiddled with the training recipe too much for this example 😄 a bunch of things could probably be changed to further optimize the results, I just wanted to keep it simple.
Lmk if you want any changes to be applied to the training example.
|
Thanks for the feedback! Wasn't clear if you wanted to add CIFAR-10 to |
Checklist
run-checks allscript has been executed.Related Issues/PRs
Closes #1132
Changes
ImageFolderDatasetto load images from disk (following a folder structure)custom-image-datasetexample that downloads CIFAR-10 and trains a simple CNN model using theImageFolderdatasetTesting
Added a couple of unit tests and an example.