Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Micromanager: *_metadata.txt and 51123 tag parsing #2226

Merged
merged 7 commits into from Feb 5, 2016

Conversation

melissalinkert
Copy link
Member

This fixes several issues with the new datasets from Thomas Julou - see https://github.com/openmicroscopy/data_repo_config/pull/76 for configuration and gh-2213 for discussion.

For each of the four new datasets, check that showinf on the *metadata.txt file shows images without an error (with this change). Without this change, showinf on test_sep/Pos0/metadata.txt should work, but all others will throw an exception.

For test2_sep, test_stack, and test2_stack, check each image (this may be easier in ImageJ) and verify that each image is unique and not blank.

For test_stack and test2_stack, the 51123 tag should be parsed for each plane. Compare the original metadata tables with and without this change; there should be several new Plane #... keys with this change.

@sbesson
Copy link
Member

sbesson commented Feb 4, 2016

Tested the current behavior using the Bio-formats 5.1.7 command-line tools and the provided dataset. The following commands:

$ showinf -autoscale test/test2_sep/Pos0/metadata.txt 
$ showinf -autoscale test/test_stack/test_stack_MMStack_Pos0_metadata.txt
$ showinf -autoscale test/test2_stack/MMStack_Pos0_metadata.txt 

all fail with a loci.formats.UnknownFormatException as described in this PR.

Using the command-line tools with this PR included, running showinf on each metadata file successfully opens a viewer where each plane is unique. Each fileset lists the metadata.txt file in the used files and the extended metadata:

$ ./bftools/showinf -no-upgrade -autoscale test/test2_stack/MMStack_Pos0_metadata.txt 
Checking file format [Micro-Manager]
Initializing reader
MicromanagerReader initializing test/test2_stack/MMStack_Pos0_metadata.txt
...
Reading core metadata
filename = test/test2_stack/MMStack_Pos0_metadata.txt
Used files:
    /Users/sbesson/Downloads/test/test2_stack/MMStack_Pos0_metadata.txt
    /Users/sbesson/Downloads/test/test2_stack/MMStack_Pos0.ome.tif
...
Reading global metadata

Reading metadata
...
MicroManagerVersion: 2.0.0-beta3-20160124
PixelSize_um: 1
PixelType: GRAY16
Plane #0 04: 31 +0100
Plane #0 Autofocus-Description: Demo auto-focus adapter
Plane #0 Autofocus-HubID: No
Plane #0 Autofocus-Name: DAutoFocus
Plane #0 BitDepth: 16

Also tested in OMERO by importing the four filesets. Without this PR, the command-line import succeeds with the following summary:

==> Summary
15 files uploaded, 7 filesets created, 7 images imported, 0 errors in 0:00:12.392

With this PR included, all files are uploaded and 4 images are created at import as expected:

==> Summary
18 files uploaded, 4 filesets created, 4 images imported, 0 errors in 0:00:06.819

and the extended metadata is correctly exposed in the Metadata tab.

screen shot 2016-02-04 at 09 21 40
screen shot 2016-02-04 at 09 21 28

The repository subset test job successfully passes on all Micro-manager filesets in the data repository. However, this PR causes issues on the full repository job when creating the list of files. Additionally, it causes issues when running tests against our OME-TIFF samples.

@melissalinkert
Copy link
Member Author

Fix pushed for the exception exposed by https://ci.openmicroscopy.org/view/Failing/job/BIOFORMATS-DEV-merge-full-repository/62/console. The subset job failures appear to be as a result of gh-2229, so not addressed here.

@sbesson
Copy link
Member

sbesson commented Feb 5, 2016

https://ci.openmicroscopy.org/job/BIOFORMATS-DEV-merge-repository-subset/107 validates this PR (independently of gh-2229). Merging this for rebasing onto dev_5_1 /cc @bramalingam.

As a documentation follow-up, the content of datasetDescription in MicromanagerReader should be modified and propagated to the autogenerated documentation to match the new semantics. Do we also need to expand the content https://www.openmicroscopy.org/site/support/bio-formats5.1/formats/micro-manager.html to explain the difference between the various datasets? /cc @hflynn

sbesson added a commit that referenced this pull request Feb 5, 2016
Micromanager: *_metadata.txt and 51123 tag parsing
@sbesson sbesson merged commit 6040fde into ome:develop Feb 5, 2016
@melissalinkert
Copy link
Member Author

--rebased-to #2232

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants