Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create NPZ file for training #99

Merged
merged 9 commits into from Jul 20, 2020
Merged

Create NPZ file for training #99

merged 9 commits into from Jul 20, 2020

Conversation

ngreenwald
Copy link
Collaborator

Basic functionality for build.py to create an NPZ with specified overlap and crop size.

caliban_toolbox/build.py Outdated Show resolved Hide resolved
@ngreenwald ngreenwald requested a review from willgraf June 8, 2020 04:53

Args:
npz_list: list of NPZ files to combine
resize_ratios: ratio used to resize each NPZ if data is of different resolutions
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this meant to be a list or an array? does 1 indicate no scaling required?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is a list that matches the NPZ files being specified, where 1 indicates no resizing required

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be a threshold for scaling. (e.g. 1.01 does not need to re-scaled)

@MekWarrior
Copy link
Collaborator

@ngreenwald I think we need to take another look at this PR. It seems somewhat fragile as presently constructed and heavily tailored to static image set. There also appears to be several locations in the code that call for padding on top of existing padding, is that by design?

@ngreenwald
Copy link
Collaborator Author

For sure, I actually have no idea what the NPZ file for training a tracking network looks like, so I'm guessing as is this won't work for that. Happy to get advice on how to make it more generalizable, or if we should explicitly restrict to static?

Yes, the two paddings accomplish different goals. The first padding is to make the full image size an even multiple of the desired crop size for the network. This is because the deepcell_toolbox.tile_image function will index from the end of the image to make up for any non-even division, which isn't the desired behavior since it will override the user-specified stride ratio.

The second padding is for data augmentation: it optionally adds blank padding equal to half of a crop size on either side of the image to squeeze an extra crop out of each row/col of the image.

@MekWarrior MekWarrior added the wip Work in Progress label Jun 12, 2020
@ngreenwald
Copy link
Collaborator Author

Hey @MekWarrior, what do we need to do to merge this in?

requirements.txt Outdated Show resolved Hide resolved
requirements.txt Outdated Show resolved Hide resolved
caliban_toolbox/build.py Outdated Show resolved Hide resolved
@MekWarrior MekWarrior merged commit cd56fac into master Jul 20, 2020
@MekWarrior MekWarrior deleted the build_npz_file branch July 20, 2020 19:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wip Work in Progress
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants