-
Notifications
You must be signed in to change notification settings - Fork 101
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for collapsing unique light paths #3261
Conversation
This new method will reduce the number of LightPath objects and by extension LogicalChannel objects when importing large plates with these metadata populated.
--rebased-to #3264 |
I assume the same changes are needed here as #3264 (comment) ? |
@joshmoore: It dropped of my radar. I will add it to my todo list. |
PR now opened. |
Now synchronised with work from @jburel. |
Code looks like it doing the right thing with the LightPaths, little less sure with the Channel/LogicalChannel bits as the schema only had Channel, but if the tests are green I would assume you are good to go. |
Thanks, all. |
Support for collapsing unique light paths
Logic and unit test cases for collapsing unique light paths. Test cases were also expanded to cover all other existing collapse logic. Some existing, but old and non-functional, test cases have been ported to TestNG and the entire suite of
ome.formats.*
unit tests added to the server unit test suite.Due to the amount of white space changes that were required, adding
?w=1
to the end of the GitHub "Files Changed" page may be beneficial during review.Manual testing has been performed:
LightPath
metadata populated (Flex in particular is useful here)LogicalChannel
andLightPath
reduction has taken placeLightPath
andLogicalChannel
reduction can be confirmed by examining the import log for a sufficiently small number. For example:Example datasets contained 12 fields, 384 wells and 2 channels resulting in 9216 unique logical channels per Plate. Completely broken on import. With a 2GB heap, takes ~4.5 hours just to get to pixel data processing:
Reduced version with just one Well's worth of
.flex
files:After reduction with just one Well's worth of
.flex
files:Total time to import with a 2GB heap is now under <60 minutes.