-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[GSK-2355, GSK-2349, GSK-2373] DataLoader structure #5
Conversation
GSK-2349 Caching
Ideas:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall great work @Hartorn!
I had doubts about promoting the class-based (as opposed to a method-based) approach to define augmented-datasets (and soon sliced-datasets) but atm I can't see any downsides as the class-based one allows for more customisability and flexibility.
I mainly did some refactoring and conflict resolutions with main.
In particular:
- I removed
facial_part
as a parameter tomodel.predict()
as this should be dictated by the passed dataset. - I renamed the base classes into:
- DataIteratorBase: The base class for all dataloader classes serving as skeleton
- DataLoaderBase: The base class for all dataloader classes that implement loading methods
- DataLoaderWrapper: The base class for all dataloader classes that serve as wrappers.
I organised the spin-off like DataLoader300W into loaders and CroppedDataLoader into wrappers.
To me it was the clearest way for now, let me know if you have other ideas.
TODO:
- Could you please remove the introduced
ground_truths
fromPredictionResult
? Even-though it might make things easier in some cases, but we should avoid mixing data-related attributes with model ones. - clean up commented code + write docstrings
- I can take care of ensuring that all notebooks are updated before we merge.
TODO In a different PR:
- We should refactor the batching and sampling that @pierlj done in https://github.com/Giskard-AI/loreal-poc/pull/3/files -- but no need to do it in this branch, let's try merging this one asap.
- unit-tests
…I/loreal-poc into dataset-first
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
More or less minor comments
GSK-2373 meta data handling in dataloader
|
No description provided.