Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multi-folder support #77

Closed
8 tasks
YipengHu opened this issue Jun 26, 2020 · 2 comments · Fixed by #80
Closed
8 tasks

Multi-folder support #77

YipengHu opened this issue Jun 26, 2020 · 2 comments · Fixed by #80
Assignees
@YipengHu @mathpluscode
Labels
feature Issues related to new feature or functionalities.
Milestone
Pre-alpha-0-loader

Comments

@YipengHu
Copy link
Member

YipengHu commented Jun 26, 2020 •
edited
Loading

Issue description

Additional support to enable training data can be sampled from multiple folders.

Type of Issue

Please delete options that are not relevant.

  • New feature request
  • Documentation update
  • Test request
  • Linting request

What's the expected result?

  • multiple folders can be specified in yaml config file;
  • the training data from all these folders will be considered (as though they are coming from the same single folder)
  • options in config files that can specify training, validation, test dataset folders
  • optionally, the multi-folder support can be extended to validation and testfolders

What's the actual result?

Additional details / screenshot
@YipengHu YipengHu added the feature Issues related to new feature or functionalities. label Jun 26, 2020
@YipengHu YipengHu added this to the Pre-alpha-0-loader milestone Jun 26, 2020
@YipengHu
Copy link
Member Author

let me know when you near finish this, I can edit the relevant doc/tutorials to be consistent with your implementation. @mathpluscode

@YipengHu YipengHu self-assigned this Jun 26, 2020
@YipengHu
Copy link
Member Author

this is also related to #41 and #36

@YipengHu YipengHu mentioned this issue Jun 27, 2020
Closed
7 tasks
mathpluscode added a commit that referenced this issue Jun 28, 2020
@mathpluscode
issue #77 rewrite h5 file loader, do not load all data into the memory
603c2dd
@mathpluscode mathpluscode mentioned this issue Jun 28, 2020
Merged
9 tasks
mathpluscode added a commit that referenced this issue Jun 28, 2020
@mathpluscode
issue #77 add set_group_structure
53a43c5
mathpluscode added a commit that referenced this issue Jun 28, 2020
@mathpluscode
issue #77 simplify h5 file loader test
c759503
mathpluscode added a commit that referenced this issue Jun 28, 2020
@mathpluscode
issue #77 move functions
a6ccd95
mathpluscode added a commit that referenced this issue Jun 28, 2020
@mathpluscode
issue #77 define common args
0e29204
mathpluscode added a commit that referenced this issue Jun 28, 2020
@mathpluscode
issue #77 add concatenated data loader
0ca72d7
mathpluscode added a commit that referenced this issue Jun 28, 2020
@mathpluscode
issue #77 specify data path for each mode
4e95b8f
mathpluscode added a commit that referenced this issue Jun 28, 2020
@mathpluscode
issue #77 add multi folder test
fd6423c
mathpluscode added a commit that referenced this issue Jun 28, 2020
@mathpluscode
issue #77 split test configs and support multiple configs
546e745
mathpluscode added a commit that referenced this issue Jun 28, 2020
@mathpluscode
issue #77 remove unused configs
4bbf976
mathpluscode added a commit that referenced this issue Jun 28, 2020
@mathpluscode
issue #77 add test to cover config merge
33c3f6a
mathpluscode added a commit that referenced this issue Jun 28, 2020
@mathpluscode
issue #77 move data config checks
0f45d60
mathpluscode added a commit that referenced this issue Jun 28, 2020
@mathpluscode
issue #77 support no validation set training
0dc9797
mathpluscode added a commit that referenced this issue Jun 28, 2020
@mathpluscode
issue #77 use image loss for all tests
995937c
mathpluscode added a commit that referenced this issue Jun 28, 2020
@mathpluscode
issue #77 fix moving image shape
14ea7da
mathpluscode added a commit that referenced this issue Jun 28, 2020
@mathpluscode
issue #77 add metrics
826ec67
mathpluscode added a commit that referenced this issue Jun 28, 2020
@mathpluscode
issue #77 change demo config
8c9c455
mathpluscode added a commit that referenced this issue Jun 28, 2020
@mathpluscode
issue #77 fix unit test
3b7978e
s-sd pushed a commit that referenced this issue Jul 2, 2020
@mathpluscode
issue #77 rewrite h5 file loader, do not load all data into the memory
aaf2bac
s-sd pushed a commit that referenced this issue Jul 2, 2020
@mathpluscode
issue #77 add set_group_structure
e95b256
s-sd pushed a commit that referenced this issue Jul 2, 2020
@mathpluscode
issue #77 simplify h5 file loader test
f5a86c3
s-sd pushed a commit that referenced this issue Jul 2, 2020
@mathpluscode
issue #77 move functions
9a03528
s-sd pushed a commit that referenced this issue Jul 2, 2020
@mathpluscode
issue #77 define common args
524cc34
s-sd pushed a commit that referenced this issue Jul 2, 2020
@mathpluscode
issue #77 add concatenated data loader
34d3afb
s-sd pushed a commit that referenced this issue Jul 2, 2020
@mathpluscode
issue #77 specify data path for each mode
ba638f5
s-sd pushed a commit that referenced this issue Jul 2, 2020
@mathpluscode
issue #77 add multi folder test
fb18bd5
s-sd pushed a commit that referenced this issue Jul 2, 2020
@mathpluscode
issue #77 split test configs and support multiple configs
5ce9e99
s-sd pushed a commit that referenced this issue Jul 2, 2020
@mathpluscode
issue #77 remove unused configs
f8600a1
s-sd pushed a commit that referenced this issue Jul 2, 2020
@mathpluscode
issue #77 add test to cover config merge
85915f4
s-sd pushed a commit that referenced this issue Jul 2, 2020
@mathpluscode
issue #77 move data config checks
88d1118
s-sd pushed a commit that referenced this issue Jul 2, 2020
@mathpluscode
issue #77 support no validation set training
bb57172
s-sd pushed a commit that referenced this issue Jul 2, 2020
@mathpluscode
issue #77 use image loss for all tests
210649d
s-sd pushed a commit that referenced this issue Jul 2, 2020
@mathpluscode
issue #77 fix moving image shape
af7d51a
s-sd pushed a commit that referenced this issue Jul 2, 2020
@mathpluscode
issue #77 add metrics
7194746
s-sd pushed a commit that referenced this issue Jul 2, 2020
@mathpluscode
issue #77 change demo config
c6324c9
s-sd pushed a commit that referenced this issue Jul 2, 2020
@mathpluscode
issue #77 fix unit test
bd6fcd3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@YipengHu YipengHu

@mathpluscode mathpluscode

Labels
feature Issues related to new feature or functionalities.
Projects
None yet
Milestone
Pre-alpha-0-loader
Development

Successfully merging a pull request may close this issue.

77 multi folder support DeepRegNet/DeepReg
2 participants
@YipengHu @mathpluscode