Include BioFormats channel name, into the ImagePlus slice label #2670
Conversation
The original code was causing javac to throw an AssertionError. Working around the labelled break (by using a flag) seems to fix this fine. Trying to minimize differences against the original code.
Fix wrong number of channels for images in fake SPW (fix #12501) (rebased onto dev_5_0)
The Travis build seems to be failing on a bug which has been fixed in the 1.2.2 version of Sphinx that we currently use in production
Use new latest/bio-formats5.0 redirects
jj2000: Fix compile on Java 8 (dev_5_0)
…the position count
Channel names, emission wavelengths, exposure times, and physical positions are now more completely recorded in MetadataStore.
Reduces the number of skips, and removes System.arraycopy when a very small number of bytes are being copied.
This should prevent issues with running 1.8-compiled classes on 1.6 or 1.7.
With this commit, we use the latest version of native-lib-loader which includes an upstream NPE fix in BaseJniExtractor.deleteLeftoverFiles().
Update version numbers to 5.0.8-SNAPSHOT
Add version history for 5.0.8
Update POM version numbers to 5.0.8
Tag version 5.0.8
Tag version 5.1.1
Tag version 5.1.2
Tag version 5.1.3
Tag version 5.1.4
…ice label. constructSliceLabel() previously created the slice label based upon the BioFormats time, position, z position and channel position and then appended the image name, where present. For example a generated slice name might look like: "c:1/4 z:2/3 t:1/3 - ExImageName" This meant that the image name was replicated for every channel and in any case is redundant since the image name is also present in the ImagePlus image title (displayed in the ImagePlus window title). Additionally any channel names specified in the BioFormats file was simply ignored. This fix does two things 1. If the BioFormat image file has a non empty channel name defined for a channel, the slice label uses this in preference to the c:i/j channel position specifier. This seems reasonable as it provides extra information and ImageJ displays the relative channel position anyway. 2. Omits the appending of the image name to every image slice (avoiding needless replication of information that is already present with ImagePlus.getTitle().
|
Dear Michael, Thank you for submitting your pull request. The current PR works great for its intended functionality. We were able to test the same with a range of multidimensional files and it works as expected. This also exposed a few more user workflows where this change could potentially be a breaking change, but we also think that the current PR is opening up a few possibilities for handling the available metadata in the slice labels. Here are a few comments/suggestions to consider, The default implementation in Bio-Formats might be used by users to access the image name via the slice label and this change might break a few macros/current workflows.This suggests that the functionality of the PR might need some additional options to make it backwards compatible as well. Having reviewed a range of multidimensional images, populating the slice-labels with channel names is definitely useful but there could be other user workflows as well, e.g.: display the relative positions (in microns,etc) of the z-planes, or time points information in a multi-dimensional (single channel) image. This points at looking for a user-configurable solution which is easy to use and is potentially accessible via the ImageJ macro language as well. User configurable option for backwards compatibility: Adding a checkbox and a dropdown menu (with options such as : default(image name), channel name, z-slices, time-point, etc…) in the Bio-Formats Importer window, could be one of the options. This will give the user more options at the import stage, in terms of controlling the population of slice labels in the ImagePlus object and will act as a backwards compatible, non-breaking change. Template PR: The following PR handles a similar scenario with ROIs and overlays in ImageJ, and the same philosophy could be used in this case as well (it’s user configurable), Please consider these suggestions and let us know what you feel about the same. Best, |
|
Dear Balaji,
Thank you for your response to my PR.
Regarding backward compatibility and the UI extension you put forward, my thoughts are as follows:
The existing code that sets all the ImagePlus slice labels for all channels to be based upon the image name appended to the c/cn (relative channel) is somewhat unintuitive and replicates data multiple times the same information that can be got from interrogating information that is already present in the ImagePlus title.
Getting the image name from the slice label is not only counterintuitive but also involves having to strip the relative channel/position/time slot information from the slice label string, so for the purpose of maintaining macro backwards compatibility, it would seem to mean maintaining a bad design.
To me the ImagePlus slice labels, where present in 3, 4, or 5D images most naturally map to the BioFormats channel names. Can anyone suggest why this assumption might be a bad one?
Adding a large set of UI check boxes to determine the construction of the slice labels, would I think, add even more confusion to what is already a quite busy/complex UI. Further, if the information gleaned from the BioFormats file is to be available in all it’s richness then I suspect that attempting to squeeze it into ImagePlus slice labels is not the most robust way of doing this.
Is there not an extensible part to an ImagePlus than can be populated with the BioFormats metadata?
Would it not be more robust if the complete BioFormats metadata could be available for inspection by some auxiliary mechanism that can be accessed, indirectly if necessary, from the ImagePlus
This certainly would appeal to me as I wish to create an Image acquisition application that will populate the XML on an OME.TIFF file with information that is core , such as channel names, physical pixel sizes etc, but would also like to add bespoke information(as BioFormat annotations) that I could conveniently and robustly access in any ImageJ Plugins that I may wish to write. I could of course write my own Plugin to do the importing but that would seem like reinventing the wheel.
— Michael Ellis
On 30 Nov 2016, at 14:01, Balaji Ramalingam ***@***.***> wrote:
Dear Michael,
Thank you for submitting your pull request.
The current PR works great for its intended functionality. We were able to test the same with a range of multidimensional files and it works as expected. This also exposed a few more user workflows where this change could potentially be a breaking change, but we also think that the current PR is opening up a few possibilities for handling the available metadata in the slice labels.
Here are a few comments/suggestions to consider,
The default implementation in Bio-Formats might be used by users to access the image name via the slice label and this change might break a few macros/current workflows.This suggests that the functionality of the PR might need some additional options to make it backwards compatible as well.
Having reviewed a range of multidimensional images, populating the slice-labels with channel names is definitely useful but there could be other user workflows as well, e.g.: display the relative positions (in microns,etc) of the z-planes, or time points information in a multi-dimensional (single channel) image. This points at looking for a user-configurable solution which is easy to use and is potentially accessible via the ImageJ macro language as well.
User configurable option for backwards compatibility: Adding a checkbox and a dropdown menu (with options such as : default(image name), channel name, z-slices, time-point, etc…) in the Bio-Formats Importer window, could be one of the options. This will give the user more options at the import stage, in terms of controlling the population of slice labels in the ImagePlus object and will act as a backwards compatible, non-breaking change.
Template PR: The following PR handles a similar scenario with ROIs and overlays in ImageJ, and the same philosophy could be used in this case as well (it’s user configurable),
#2333 <#2333>
Uses the ImporterOptions class : http://downloads.openmicroscopy.org/bio-formats/5.2.4/api/loci/plugins/in/class-use/ImporterOptions.html <http://downloads.openmicroscopy.org/bio-formats/5.2.4/api/loci/plugins/in/class-use/ImporterOptions.html>
Please consider these suggestions and let us know what you feel about the same.
Best,
Balaji
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <#2670 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEbq95HX_2WWXgnftCCwQiD05sbt8Qaxks5rDYGogaJpZM4K5b-j>.
Michael Ellis (Managing Director)
Digital Scientific UK Ltd.
http://www.dsuk.biz <http://www.dsuk.biz/>
michael.ellis@dsuk.biz
tel: +44(0)1223 911215
The Commercial Centre
6 Green End
Cambridge
CB23 7DY
|
|
Dear Michael,
As mentioned above, the assumptions beyond your naming proposal are certainly sensible. There are a few reasons why we need to take its impact on the community into account. At the moment, we are dealing with a similar issue as a result of enabling what we considered as being the correct behavior while reading pyramidal Zeiss CZI images. These changes turned out to affect existing users workflows we were not aware of (#2634) and we are currently working on making these changes backwards compatible in Bio-Formats 5.3.0. Also, we can imagine experiment types and imaging modalities where the most natural labelling might be the timestamp, XY stage position, etc. For all these reasons, we think that proposing and documenting these types of changes in a major release where we allow ourselves to break the API is the safest policy with regard to our community.
Agreed on preventing unnecessary complexity on the existing UI.
While we certainly have a ticket to review this interface [1], I see two possibilities in the existing API that might allow you to access the metadata:
[1] https://trac.openmicroscopy.org/ome/ticket/13306 |
|
On 5 Dec 2016, at 10:24, Sébastien Besson ***@***.***> wrote:
Dear Michael,
Regarding backward compatibility and the UI extension you put forward, my thoughts are as follows:
The existing code that sets all the ImagePlus slice labels for all channels to be based upon the image name appended to the c/cn (relative channel) is somewhat unintuitive and replicates data multiple times the same information that can be got from interrogating information that is already present in the ImagePlus title.
Getting the image name from the slice label is not only counterintuitive but also involves having to strip the relative channel/position/time slot information from the slice label string, so for the purpose of maintaining macro backwards compatibility, it would seem to mean maintaining a bad design.
To me the ImagePlus slice labels, where present in 3, 4, or 5D images most naturally map to the BioFormats channel names. Can anyone suggest why this assumption might be a bad one?
As mentioned above, the assumptions beyond your naming proposal are certainly sensible. There are a few reasons why we need to take its impact on the community into account. At the moment, we are dealing with a similar issue as a result of enabling what we considered as being the correct behavior while reading pyramidal Zeiss CZI images. These changes turned out to affect existing users workflows we were not aware of (#2634 <#2634>) and we are currently working on making these changes backwards compatible in Bio-Formats 5.3.0. Also, we can imagine experiment types and imaging modalities where the most natural labelling might be the timestamp, XY stage position, etc.
For all these reasons, we think that proposing and documenting these types of changes in a major release where we allow ourselves to break the API is the safest policy with regard to our community.
Agreed the API changes differed to amor release change is safest option. Next question obviously (for a newbie), what is the timescale for next major release? :-) It’s not pressure, that this is open source and I can have my own BioFormats Importer that does it the way I want is an acceptable workaround.
Adding a large set of UI check boxes to determine the construction of the slice labels, would I think, add even more confusion to what is already a quite busy/complex UI. Further, if the information gleaned from the BioFormats file is to be available in all it’s richness then I suspect that attempting to squeeze it into ImagePlus slice labels is not the most robust way of doing this.
Agreed on preventing unnecessary complexity on the existing UI.
Is there not an extensible part to an ImagePlus than can be populated with the BioFormats metadata?
Would it not be more robust if the complete BioFormats metadata could be available for inspection by some auxiliary mechanism that can be accessed, indirectly if necessary, from the ImagePlus
While we certainly have a ticket to review this interface [1], I see two possibilities in the existing API that might allow you to access the metadata:
the ImagePlusReader fills OME-XML metadata in the description field of the FileInfo of the ImagePlus [2]
if you are using an ImportProcess before ImagePlusReader, this object should give you some introspection on the Bio-Formats readers and allow you to access the full API to retrieve the metadata
Do either of these solutions allow you to move forward in the case of your application?
I am a little unclear about the second option. I like that in the first option no additional work is necessary and presumably it is fairly trivial to establish whether or not the contents of the FileInfo field is indeed OME-XML. The point being that this detaches the import processing from image interrogation. i.e. one could simply examine the FileInfo for OME-XML information and not have to care how the image had been imported.
I do wonder though if using the FileInfo field is overloading it’s functionality. It’s a shame that the ImagePlus does not have a <Map, String> properties field that could be indexed with domain specific strings such the anyone could attach additional info to an ImagePlus without fear of screwing up anyone else’s additions.
[1] https://trac.openmicroscopy.org/ome/ticket/13306 <https://trac.openmicroscopy.org/ome/ticket/13306>
[2] https://github.com/openmicroscopy/bioformats/blob/v5.2.4/components/bio-formats-plugins/src/loci/plugins/in/ImagePlusReader.java#L515 <https://github.com/openmicroscopy/bioformats/blob/v5.2.4/components/bio-formats-plugins/src/loci/plugins/in/ImagePlusReader.java#L515>
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <#2670 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEbq93R1LFBrNtOQrytNAYdPaqf5Kym9ks5rE-ZVgaJpZM4K5b-j>.
Many thanks and best regards — Michael Ellis
Michael Ellis (Managing Director)
Digital Scientific UK Ltd.
http://www.dsuk.biz <http://www.dsuk.biz/>
michael.ellis@dsuk.biz
tel: +44(0)1223 911215
The Commercial Centre
6 Green End
Cambridge
CB23 7DY
|
|
Hi Michael, thanks for asking. We have no concrete roadmap for the next set of breaking changes yet but once we are done with Bio-Formats 5.3.0, we will certainly review our targets for 2017. Once we have something more public, we will make sure this gets recorded alongside the list of other proposed changes. Glad to hear the FileInfo workaround would suit your need and I agree this might be the most generic approach. The usage of the Validating the string should be doable and a good starting point might be the OMEXMLService.validateOMEXML(String) API. |
|
Closing this as we won't include this in the stable 5.x series. Added a card to the next breaking column (6.x) to come back to it. |
|
I am not sure what "Added a card to the next breaking column (6.x) to come back to it." but can someone tell me if this pull request has been forgotten about or if I can track it's progress in the context of this (6.x) project? |
|
The card is at https://trello.com/c/KHxM1IIe/456-imagej-imageplus-slicelabel-modification: subscribe to that to track progress. Sorry for the confusion! Update: Sorry, that board does not seem to be publicly visible. I am sure a better answer will be forthcoming but in the meantime please rest assured that your work is not forgotten. |
|
When I click on the link I get:
————
Card not found.
This card may be on a private board. If someone gave you this link, they may need to invite you to one of their boards or teams.
————
I have just creates a new Trello account and my user id is Michael Ellis (michaelellis52)
Any help greatly appreciated.
… On 23 Jun 2017, at 12:09, Mark Carroll ***@***.***> wrote:
The card is at https://trello.com/c/KHxM1IIe/456-imagej-imageplus-slicelabel-modification <https://trello.com/c/KHxM1IIe/456-imagej-imageplus-slicelabel-modification>: subscribe to that to track progress. Sorry for the confusion!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <#2670 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEbq90ZSpfoFjJpKuNpbGkrTsf2PNPM6ks5sG5zogaJpZM4K5b-j>.
|
|
Can someone please help me out.
The Trello link just gives me an error and I really would like to track the progress on this BioFormats ImageJ Plugin bug fix/feature. I went to some considerable length to learn about all the tools necessary to make the original pull request which in the end was not used for BioFormat point release.
BioFormats looks to be an ideal match to our requirements for saving XYCZT images from our imaging application and I am keen to use public standards. But with what seems to be sketchy documentation and little support I have to consider carefully whether BioFormats will be stable, supported and safe for our customers.
Another post I made to this group about how to store thumbnails with an image has received to reply. I understand this is open source and free. I want in! I’m happy to contribute if there is anything I can. But I just cannot seem to break in!
Thanking anyone in anticipation of any help offered.
— Michael Ellis
On 23 Jun 2017, at 12:15, Michael Ellis ***@***.***> wrote:
When I click on the link I get:
————
Card not found.
This card may be on a private board. If someone gave you this link, they may need to invite you to one of their boards or teams.
————
I have just creates a new Trello account and my user id is Michael Ellis (michaelellis52)
Any help greatly appreciated.
> On 23 Jun 2017, at 12:09, Mark Carroll ***@***.*** ***@***.***>> wrote:
>
> The card is at https://trello.com/c/KHxM1IIe/456-imagej-imageplus-slicelabel-modification <https://trello.com/c/KHxM1IIe/456-imagej-imageplus-slicelabel-modification>: subscribe to that to track progress. Sorry for the confusion!
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub <#2670 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEbq90ZSpfoFjJpKuNpbGkrTsf2PNPM6ks5sG5zogaJpZM4K5b-j>.
>
Michael Ellis (Managing Director)
Digital Scientific UK Ltd.
http://www.dsuk.biz <http://www.dsuk.biz/>
michael.ellis@dsuk.biz
tel: +44(0)1223 911215
The Commercial Centre
6 Green End
Cambridge
CB23 7DY
=== END OF EMAIL ===
|
|
Hi Michael,
Thanks for the time invested and driving the interaction via this Pull Request. As mentioned in the thread above, the intent was completely understood and sensible but the proposed implementation included breaking behavioral changes which was not suitable for a point or a minor version increment. The card mentioned above was originally captured for the next round of breaking change in a private backlog board. Thanks for raising the access problem, we will review this internally. We are currently defining the roadmap for our next minor version release i.e. Bio-Formats 5.6.0 which will be non-breaking once more. I have migrated the card to the inbox of the
Glad to hear you are considering open standards within your imaging application. Choosing a low-level library is an important commitment and absolutely requires serious scoping. From our side, the expectation is that the stability and support of Bio-Formats is as strong as ever and we are not aware of any security concern as of today. If you have more specifics about which portions of the OME Model and Formats and/or Bio-Formats documentation could be improved and how, we would be happy to hear about it.
I am cross-linking to the relevant thread for reference. Our general policy is to treat all community input including mailing lists, forums, issues within the next working day. We just answered this particular thread giving an update on what is and is not possible with the current specification and API. More generally, the public community tools like GH issues, mailing lists are definitely the primary ways to interact with the team and start these complex discussions. We are watching these threads and sometimes a delay means we are discussing it internally or working towards an appropriate answer to elaborate on the existing technology. |
|
Sébastien,
Many thanks for your reply!
In respect of the the public board <https://trello.com/b/Jb2oR35f/bio-formats-560>, I can now see it but there seems no facility for me to comment. That may be intentional fi the board is read/write for those that develop.
In respect of backwards compatible, my opinion is that whilst mechanisms could dreamt up that would allow for the co-existence of the existing behaviour (definitely rather questionable), the behaviour my PR puts forward, and/or any other behaviour, I *think* preserving the existing behaviour, where the information redundancy looks like a bad design, to avoid breaking an unknown number of scripts (surely small if any at all), would come at at a greater cost from going with a cleaner design. But this of course is an uninformed opinion - I have no way of knowing just how many scripts may depend on the current slice naming convention.
I am, for now, persisting with the use of OME_TIFF as our default image storage format. There will almost certainly be more questions I will need to post in the future. My apologies in advance!
Again thanks!
— Michael Ellis
… On 26 Jun 2017, at 15:44, Sébastien Besson ***@***.***> wrote:
Hi Michael,
The Trello link just gives me an error and I really would like to track the progress on this BioFormats ImageJ Plugin bug fix/feature. I went to some considerable length to learn about all the tools necessary to make the original pull request which in the end was not used for BioFormat point release.
Thanks for the time invested and driving the interaction via this Pull Request. As mentioned in the thread above, the intent was completely understood and sensible but the proposed implementation included breaking behavioral changes which was not suitable for a point or a minor version increment.
The card mentioned above was originally captured for the next round of breaking change in a private backlog board. Thanks for raising the access problem, we will review this internally.
We are currently defining the roadmap for our next minor version release i.e. Bio-Formats 5.6.0 which will be non-breaking once more. I have migrated the card to the inbox of the
public board <https://trello.com/b/Jb2oR35f/bio-formats-560> for discussion. Our major question is whether we can find a backwards-compatible solution that would address your problem and make this API extensible for new implementations in the future.
BioFormats looks to be an ideal match to our requirements for saving XYCZT images from our imaging application and I am keen to use public standards. But with what seems to be sketchy documentation and little support I have to consider carefully whether BioFormats will be stable, supported and safe for our customers.
Glad to hear you are considering open standards within your imaging application. Choosing a low-level library is an important commitment and absolutely requires serious scoping. From our side, the expectation is that the stability and support of Bio-Formats is as strong as ever and we are not aware of any security concern as of today.
If you have more specifics about which portions of the OME Model and Formats <https://www.openmicroscopy.org/site/support/ome-model/> and/or Bio-Formats <https://www.openmicroscopy.org/site/support/bio-formats5/> documentation could be improved and how, we would be happy to hear about it.
Another post I made to this group about how to store thumbnails with an image has received to reply. I understand this is open source and free. I want in! I’m happy to contribute if there is anything I can. But I just cannot seem to break in!
I am cross-linking to the relevant thread <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/2017-June/003995.html> for reference. Our general policy is to treat all community input including mailing lists, forums, issues within the next working day <https://www.openmicroscopy.org/site/support/contributing/team-communication.html#forums-and-mailing-list>. We just answered this particular thread giving an update on what is and is not possible with the current specification and API.
More generally, the public community tools like GH issues, mailing lists are definitely the primary ways to interact with the team and start these complex discussions. We are watching these threads and sometimes a delay means we are discussing it internally or working towards an appropriate answer to elaborate on the existing technology.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <#2670 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEbq96NDzKUEAkTDjWqSWsGKa6EA0ernks5sH8PMgaJpZM4K5b-j>.
|
This pull requests adds functionality to the Bio-Formats Importer to include any BioFormats channel names, where present, into the ImagePlus slice labels.
constructSliceLabel() previously created the slice label based upon the BioFormats time, position, z position and channel position and then appended the image name, where present. For example a generated slice name might look like:
This meant that the image name was replicated for every channel and in any case is redundant since the image name is also present in the ImagePlus image title (displayed in the ImagePlus window title).
Additionally any channel names specified in the BioFormats file was simply ignored.
This fix does two things
If the BioFormat image file has a non empty channel name defined for a channel, the slice label uses this in preference to the c:i/j channel position specifier. This seems reasonable as it provides extra information and ImageJ displays the relative channel position anyway.
Omits the appending of the image name to every image slice (avoiding needless replication of information that is already present with ImagePlus.getTitle().