-
Notifications
You must be signed in to change notification settings - Fork 242
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bfoptions: Add ability to read options from file #3605
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 overall, just a couple of minor questions.
components/formats-api/src/loci/formats/in/DynamicMetadataOptions.java
Outdated
Show resolved
Hide resolved
Added a new commit to throw a warning if the options in the bfoptions file are incorrect. This is one possible implementation of the requested functionality as discussed in the last formats meeting, initially using CZI as an example. |
Some preliminary functional testing of the options file was performed in the context of a concrete reader - see #3606 (comment). A few additional comments:
|
See #3606 (comment) for an example of testing of the import worfklow with an options file tested the I have been looking into adding configuration files for non-regression tests using the same ScanR dataset with an options file setting
As the default behavior of the reader for this dataset is to detect 357 series, I suspect the above indicates the two readers used for generating the configuration (flattened vs non-flattened) are not seeing the same options. I will take a look at this next week-end as well as test the workflow with the new FlexReader option. Another aspect of this API is to decide how it affects serialization i.e. should a cached reader detect mismatching options and be able to invalidate itself? Overall, there are a few unanswered questions but also at the same time a few open PR which depend on this new API esp. to declare their available options. I am considering getting this merged, tag a milestone release, start consuming it in various workflows and use cases and report blockers or required API/implementation changes via issues. Proposing to discuss the roadmap to Bio-Formats 6.6.0 as part of Monday's formats meeting. |
As a summary of Monday's discussion, leaving one more day for a first round of feedback on this API addition. Barring objections, I will start integrating the first round of options PR so that follow-up PRs can be opened. Overall we are currently tending towards a mid-October release of Bio-Formats 6.6. |
String p = f.getParent(); | ||
String n = f.getName(); | ||
if (n.indexOf(".") >= 0) { | ||
n = n.substring(0, n.indexOf(".")); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While working on #3617, I realized one downside of this naming pattern is that the option filename is not unique. Thinking of the potential scenarios, I think the two principal ones are:
- filesets where constituting files have the same base name but different extensions e.g.
drosophila.h5, drosophila.xml
- separate filesets sharing the same basename e.g.
test.tiff, test.png
I will try and test the first case but we might want to review whether the second case is a blocker. Currently I assume it would mean both filesets would share an option file.
This is an initial prototype of the potential functionality for persisting options using a .bfoptions file as discussed at the formats meeting this week.
If a reader detects a file in the same folder as the currentId with the same name and extension
.bfoptions
then the options will be parsed from the fileAn example of an options file might be: