Zeiss CZI: fix dimension calculations when expected planes are missing #1386

melissalinkert · 2014-10-23T22:29:17Z

This should fix http://trac.openmicroscopy.org.uk/ome/ticket/12556. The dimensions for the files in QA 9521 and 10301 should now match the expected dimensions listed in the ticket and QA issues. I would not expect this to cause any test failures.

/cc @emilroz

See #12556.

emilroz · 2014-10-24T09:17:53Z

Showinf results:

QA 10301

Reading core metadata
Filename = /Users/emilrozbicki/Documents/Data/10301/141016_OP43adult_Worm3_DAPI_head.czi
Used files:
/Users/emilrozbicki/Documents/Data/10301/141016_OP43adult_Worm3_DAPI_head.czi
/Users/emilrozbicki/Documents/Data/10301/141016_OP43adult_Worm3_DAPI_head(1).czi
/Users/emilrozbicki/Documents/Data/10301/141016_OP43adult_Worm3_DAPI_head(2).czi
/Users/emilrozbicki/Documents/Data/10301/141016_OP43adult_Worm3_DAPI_head(3).czi
/Users/emilrozbicki/Documents/Data/10301/141016_OP43adult_Worm3_DAPI_head(4).czi
/Users/emilrozbicki/Documents/Data/10301/141016_OP43adult_Worm3_DAPI_head(5).czi
/Users/emilrozbicki/Documents/Data/10301/141016_OP43adult_Worm3_DAPI_head(6).czi
/Users/emilrozbicki/Documents/Data/10301/141016_OP43adult_Worm3_DAPI_head(7).czi
/Users/emilrozbicki/Documents/Data/10301/141016_OP43adult_Worm3_DAPI_head(8).czi
/Users/emilrozbicki/Documents/Data/10301/141016_OP43adult_Worm3_DAPI_head(9).czi
Series count = 5
Series #0 :
Image count = 100
RGB = false (1) 
Interleaved = false
Indexed = true (false color, 16-bit LUT: 3 x 65536)
Width = 300
Height = 300
SizeZ = 50
SizeT = 1
SizeC = 2 (1 Channel x 2 Illumination)
Thumbnail size = 128 x 128
Endianness = intel (little)
Dimension order = XYCZT (certain)
Pixel type = uint16
Valid bits per pixel = 16
Metadata complete = false
Thumbnail series = false
-----
Plane #0 <=> Z 0, C 0, T 0
Plane #48 <=> Z 24, C 0, T 0
Plane #49 <=> Z 24, C 1, T 0
Plane #50 <=> Z 25, C 0, T 0
Plane #51 <=> Z 25, C 1, T 0
Plane #52 <=> Z 26, C 0, T 0
Plane #99 <=> Z 49, C 1, T 0
Series #1 :
Image count = 100
RGB = false (1) 
Interleaved = false
Indexed = true (false color, 16-bit LUT: 3 x 65536)
Width = 300
Height = 300
SizeZ = 50
SizeT = 1
SizeC = 2 (1 Channel x 2 Illumination)
Thumbnail size = 128 x 128
Endianness = intel (little)
Dimension order = XYCZT (certain)
Pixel type = uint16
Valid bits per pixel = 16
Metadata complete = false
Thumbnail series = false
-----
Plane #0 <=> Z 0, C 0, T 0
Plane #48 <=> Z 24, C 0, T 0
Plane #49 <=> Z 24, C 1, T 0
Plane #50 <=> Z 25, C 0, T 0
Plane #51 <=> Z 25, C 1, T 0
Plane #52 <=> Z 26, C 0, T 0
Plane #99 <=> Z 49, C 1, T 0
Series #2 :
Image count = 100
RGB = false (1) 
Interleaved = false
Indexed = true (false color, 16-bit LUT: 3 x 65536)
Width = 300
Height = 300
SizeZ = 50
SizeT = 1
SizeC = 2 (1 Channel x 2 Illumination)
Thumbnail size = 128 x 128
Endianness = intel (little)
Dimension order = XYCZT (certain)
Pixel type = uint16
Valid bits per pixel = 16
Metadata complete = false
Thumbnail series = false
-----
Plane #0 <=> Z 0, C 0, T 0
Plane #48 <=> Z 24, C 0, T 0
Plane #49 <=> Z 24, C 1, T 0
Plane #50 <=> Z 25, C 0, T 0
Plane #51 <=> Z 25, C 1, T 0
Plane #52 <=> Z 26, C 0, T 0
Plane #99 <=> Z 49, C 1, T 0
Series #3 :
Image count = 100
RGB = false (1) 
Interleaved = false
Indexed = true (false color, 16-bit LUT: 3 x 65536)
Width = 300
Height = 300
SizeZ = 50
SizeT = 1
SizeC = 2 (1 Channel x 2 Illumination)
Thumbnail size = 128 x 128
Endianness = intel (little)
Dimension order = XYCZT (certain)
Pixel type = uint16
Valid bits per pixel = 16
Metadata complete = false
Thumbnail series = false
-----
Plane #0 <=> Z 0, C 0, T 0
Plane #48 <=> Z 24, C 0, T 0
Plane #49 <=> Z 24, C 1, T 0
Plane #50 <=> Z 25, C 0, T 0
Plane #51 <=> Z 25, C 1, T 0
Plane #52 <=> Z 26, C 0, T 0
Plane #99 <=> Z 49, C 1, T 0
Series #4 :
Image count = 100
RGB = false (1) 
Interleaved = false
Indexed = true (false color, 16-bit LUT: 3 x 65536)
Width = 300
Height = 300
SizeZ = 50
SizeT = 1
SizeC = 2 (1 Channel x 2 Illumination)
Thumbnail size = 128 x 128
Endianness = intel (little)
Dimension order = XYCZT (certain)
Pixel type = uint16
Valid bits per pixel = 16
Metadata complete = false
Thumbnail series = false
-----
Plane #0 <=> Z 0, C 0, T 0
Plane #48 <=> Z 24, C 0, T 0
Plane #49 <=> Z 24, C 1, T 0
Plane #50 <=> Z 25, C 0, T 0
Plane #51 <=> Z 25, C 1, T 0
Plane #52 <=> Z 26, C 0, T 0
Plane #99 <=> Z 49, C 1, T 0

QA 9521

Reading core metadata
Filename = /Volumes/ome/apache_repo/9521/c3_z11_t1.czi
Series count = 1
Series #0 :
Image count = 33
RGB = false (1) 
Interleaved = false
Indexed = true (false color, 16-bit LUT: 3 x 65536)
Width = 512
Height = 512
SizeZ = 11
SizeT = 1
SizeC = 3
Thumbnail size = 128 x 128
Endianness = intel (little)
Dimension order = XYCZT (certain)
Pixel type = uint16
Valid bits per pixel = 16
Metadata complete = false
Thumbnail series = false
-----
Plane #0 <=> Z 0, C 0, T 0
Plane #14 <=> Z 4, C 2, T 0
Plane #15 <=> Z 5, C 0, T 0
Plane #16 <=> Z 5, C 1, T 0
Plane #17 <=> Z 5, C 2, T 0
Plane #18 <=> Z 6, C 0, T 0
Plane #32 <=> Z 10, C 2, T 0

Still needs to be checked on a real upload.
Looks good for QA10301 which before fix was showing Series Count = 3. Also looks as expected for QA9521.
There is still an issue of a different number of Z planes per Series for QA10301, which is also a case for other SPIM .CZIs. Each series in 10301 has a different number of Z planes but currently they all get ascribed the highest number, in this case 50. In principle it's ok however it causes issues after uploading this data to OMERO. When "empty" planes are retrieved from RawPixelStore after planes with real data they contain rubbish. So currently RawPixelStore needs to be closed and open before getPlane() call which is not ideal as it makes the IO time at least 2x longer.

This prevents blank planes from containing garbage data.

melissalinkert · 2014-10-24T14:39:25Z

One more commit pushed which should resolve the rubbish data in empty planes issue. For QA 10301, the Z counts are being set to 50 across the board because that seems to be what Zen does - it's true sparse data, so unfortunately just reducing the Z count doesn't accurately represent the acquisition.

emilroz · 2014-10-27T12:01:03Z

Tested using OMERO.CPP and TROUT:

5.1.0-m1-284-1c19755-ice34-b406 (no changes to czi reader)
5.1.0-m1-441-1f5b2b1-ice34-b310 (new reader)

Result pre-changes:

only 3 series imported,
rubbish in the buffer when reading "EMPTY" planes (look at the plane Upgrade check updates #34 in the output)

output:

(2014-10-27.11:53:41) Connected to server: trout.openmicroscopy.org as user-1
--------
User: user-1
Id: 8
Current group: private-1
---------
Title: 141016_OP43adult_Worm3_DAPI_head.czi [141016_OP43adult_Worm3_DAPI_head #1]
Description: no description
# of Channels: 2
# of Timepoints: 1
Size X: 300 pixels; 0.287554
Size Y: 300 pixels; 0.287554
Size Z: 50 pixels; 0.579444
Filseset:
21401 141016_OP43adult_Worm3_DAPI_head.czi [141016_OP43adult_Worm3_DAPI_head #1]
21402 141016_OP43adult_Worm3_DAPI_head.czi [141016_OP43adult_Worm3_DAPI_head #2]
21403 141016_OP43adult_Worm3_DAPI_head.czi [141016_OP43adult_Worm3_DAPI_head #3]
2014-10-27.11:53:41 Reading Image: 21401 time point: 0 channel: 0
Plane: 0, timepoint: 0, channel: 0; Intensity(min,max): (150 49114)
Plane: 1, timepoint: 0, channel: 0; Intensity(min,max): (166 41492)
Plane: 2, timepoint: 0, channel: 0; Intensity(min,max): (160 38932)
Plane: 3, timepoint: 0, channel: 0; Intensity(min,max): (160 38894)
Plane: 4, timepoint: 0, channel: 0; Intensity(min,max): (168 37404)
Plane: 5, timepoint: 0, channel: 0; Intensity(min,max): (150 37550)
Plane: 6, timepoint: 0, channel: 0; Intensity(min,max): (154 54016)
Plane: 7, timepoint: 0, channel: 0; Intensity(min,max): (162 61894)
Plane: 8, timepoint: 0, channel: 0; Intensity(min,max): (134 53452)
Plane: 9, timepoint: 0, channel: 0; Intensity(min,max): (174 40448)
Plane: 10, timepoint: 0, channel: 0; Intensity(min,max): (168 42148)
Plane: 11, timepoint: 0, channel: 0; Intensity(min,max): (168 41112)
Plane: 12, timepoint: 0, channel: 0; Intensity(min,max): (168 44172)
Plane: 13, timepoint: 0, channel: 0; Intensity(min,max): (168 40898)
Plane: 14, timepoint: 0, channel: 0; Intensity(min,max): (170 47780)
Plane: 15, timepoint: 0, channel: 0; Intensity(min,max): (168 60990)
Plane: 16, timepoint: 0, channel: 0; Intensity(min,max): (172 63862)
Plane: 17, timepoint: 0, channel: 0; Intensity(min,max): (162 55598)
Plane: 18, timepoint: 0, channel: 0; Intensity(min,max): (166 42840)
Plane: 19, timepoint: 0, channel: 0; Intensity(min,max): (160 50162)
Plane: 20, timepoint: 0, channel: 0; Intensity(min,max): (168 50722)
Plane: 21, timepoint: 0, channel: 0; Intensity(min,max): (158 47470)
Plane: 22, timepoint: 0, channel: 0; Intensity(min,max): (136 55748)
Plane: 23, timepoint: 0, channel: 0; Intensity(min,max): (168 59530)
Plane: 24, timepoint: 0, channel: 0; Intensity(min,max): (168 55838)
Plane: 25, timepoint: 0, channel: 0; Intensity(min,max): (146 57462)
Plane: 26, timepoint: 0, channel: 0; Intensity(min,max): (152 65534)
Plane: 27, timepoint: 0, channel: 0; Intensity(min,max): (166 62526)
Plane: 28, timepoint: 0, channel: 0; Intensity(min,max): (170 54770)
Plane: 29, timepoint: 0, channel: 0; Intensity(min,max): (164 41348)
Plane: 30, timepoint: 0, channel: 0; Intensity(min,max): (168 37794)
Plane: 31, timepoint: 0, channel: 0; Intensity(min,max): (168 41094)
Plane: 32, timepoint: 0, channel: 0; Intensity(min,max): (158 35528)
Plane: 33, timepoint: 0, channel: 0; Intensity(min,max): (168 25998)
Plane: 34, timepoint: 0, channel: 0; Intensity(min,max): (1 65098)
Plane: 35, timepoint: 0, channel: 0; Intensity(min,max): (168 25998)
Plane: 36, timepoint: 0, channel: 0; Intensity(min,max): (1 65098)
Plane: 37, timepoint: 0, channel: 0; Intensity(min,max): (168 25998)
Plane: 38, timepoint: 0, channel: 0; Intensity(min,max): (1 65098)
Plane: 39, timepoint: 0, channel: 0; Intensity(min,max): (168 25998)
Plane: 40, timepoint: 0, channel: 0; Intensity(min,max): (1 65098)
Plane: 41, timepoint: 0, channel: 0; Intensity(min,max): (168 25998)
Plane: 42, timepoint: 0, channel: 0; Intensity(min,max): (1 65098)
Plane: 43, timepoint: 0, channel: 0; Intensity(min,max): (168 25998)
Plane: 44, timepoint: 0, channel: 0; Intensity(min,max): (1 65098)
Plane: 45, timepoint: 0, channel: 0; Intensity(min,max): (168 25998)
Plane: 46, timepoint: 0, channel: 0; Intensity(min,max): (1 65098)
Plane: 47, timepoint: 0, channel: 0; Intensity(min,max): (168 25998)
Plane: 48, timepoint: 0, channel: 0; Intensity(min,max): (1 65098)
Plane: 49, timepoint: 0, channel: 0; Intensity(min,max): (168 25998)
Closing pixel store

Results with changes to the CZI reader:

imported 5 series (as expected)
no rubbish in the buffer for "EMPTY" planes (look at the plane Upgrade check updates #34 and compare with the output above)

output:

(2014-10-27.11:55:57) Connected to server: trout.openmicroscopy.org as user-1
--------
User: user-1
Id: 8
Current group: private-1
---------
Title: 141016_OP43adult_Worm3_DAPI_head.czi [141016_OP43adult_Worm3_DAPI_head #1]
Description: no description
# of Channels: 2
# of Timepoints: 1
Size X: 300 pixels; 0.287554
Size Y: 300 pixels; 0.287554
Size Z: 50 pixels; 0.579444
Fileset:
21115 141016_OP43adult_Worm3_DAPI_head.czi [141016_OP43adult_Worm3_DAPI_head #1]
21116 141016_OP43adult_Worm3_DAPI_head.czi [141016_OP43adult_Worm3_DAPI_head #2]
21117 141016_OP43adult_Worm3_DAPI_head.czi [141016_OP43adult_Worm3_DAPI_head #3]
21118 141016_OP43adult_Worm3_DAPI_head.czi [141016_OP43adult_Worm3_DAPI_head #4]
21119 141016_OP43adult_Worm3_DAPI_head.czi [141016_OP43adult_Worm3_DAPI_head #5]
2014-10-27.11:55:57 Reading Image: 21115 time point: 0 channel: 0
Plane: 0, timepoint: 0, channel: 0; Intensity(min,max): (150 49114)
Plane: 1, timepoint: 0, channel: 0; Intensity(min,max): (166 41492)
Plane: 2, timepoint: 0, channel: 0; Intensity(min,max): (160 38932)
Plane: 3, timepoint: 0, channel: 0; Intensity(min,max): (160 38894)
Plane: 4, timepoint: 0, channel: 0; Intensity(min,max): (168 37404)
Plane: 5, timepoint: 0, channel: 0; Intensity(min,max): (150 37550)
Plane: 6, timepoint: 0, channel: 0; Intensity(min,max): (154 54016)
Plane: 7, timepoint: 0, channel: 0; Intensity(min,max): (162 61894)
Plane: 8, timepoint: 0, channel: 0; Intensity(min,max): (134 53452)
Plane: 9, timepoint: 0, channel: 0; Intensity(min,max): (174 40448)
Plane: 10, timepoint: 0, channel: 0; Intensity(min,max): (168 42148)
Plane: 11, timepoint: 0, channel: 0; Intensity(min,max): (168 41112)
Plane: 12, timepoint: 0, channel: 0; Intensity(min,max): (168 44172)
Plane: 13, timepoint: 0, channel: 0; Intensity(min,max): (168 40898)
Plane: 14, timepoint: 0, channel: 0; Intensity(min,max): (170 47780)
Plane: 15, timepoint: 0, channel: 0; Intensity(min,max): (168 60990)
Plane: 16, timepoint: 0, channel: 0; Intensity(min,max): (172 63862)
Plane: 17, timepoint: 0, channel: 0; Intensity(min,max): (162 55598)
Plane: 18, timepoint: 0, channel: 0; Intensity(min,max): (166 42840)
Plane: 19, timepoint: 0, channel: 0; Intensity(min,max): (160 50162)
Plane: 20, timepoint: 0, channel: 0; Intensity(min,max): (168 50722)
Plane: 21, timepoint: 0, channel: 0; Intensity(min,max): (158 47470)
Plane: 22, timepoint: 0, channel: 0; Intensity(min,max): (136 55748)
Plane: 23, timepoint: 0, channel: 0; Intensity(min,max): (168 59530)
Plane: 24, timepoint: 0, channel: 0; Intensity(min,max): (168 55838)
Plane: 25, timepoint: 0, channel: 0; Intensity(min,max): (146 57462)
Plane: 26, timepoint: 0, channel: 0; Intensity(min,max): (152 65534)
Plane: 27, timepoint: 0, channel: 0; Intensity(min,max): (166 62526)
Plane: 28, timepoint: 0, channel: 0; Intensity(min,max): (170 54770)
Plane: 29, timepoint: 0, channel: 0; Intensity(min,max): (164 41348)
Plane: 30, timepoint: 0, channel: 0; Intensity(min,max): (168 37794)
Plane: 31, timepoint: 0, channel: 0; Intensity(min,max): (168 41094)
Plane: 32, timepoint: 0, channel: 0; Intensity(min,max): (158 35528)
Plane: 33, timepoint: 0, channel: 0; Intensity(min,max): (168 25998)
Plane: 34, timepoint: 0, channel: 0; Intensity(min,max): (0 0)
Plane: 35, timepoint: 0, channel: 0; Intensity(min,max): (0 0)
Plane: 36, timepoint: 0, channel: 0; Intensity(min,max): (0 0)
Plane: 37, timepoint: 0, channel: 0; Intensity(min,max): (0 0)
Plane: 38, timepoint: 0, channel: 0; Intensity(min,max): (0 0)
Plane: 39, timepoint: 0, channel: 0; Intensity(min,max): (0 0)
Plane: 40, timepoint: 0, channel: 0; Intensity(min,max): (0 0)
Plane: 41, timepoint: 0, channel: 0; Intensity(min,max): (0 0)
Plane: 42, timepoint: 0, channel: 0; Intensity(min,max): (0 0)
Plane: 43, timepoint: 0, channel: 0; Intensity(min,max): (0 0)
Plane: 44, timepoint: 0, channel: 0; Intensity(min,max): (0 0)
Plane: 45, timepoint: 0, channel: 0; Intensity(min,max): (0 0)
Plane: 46, timepoint: 0, channel: 0; Intensity(min,max): (0 0)
Plane: 47, timepoint: 0, channel: 0; Intensity(min,max): (0 0)
Plane: 48, timepoint: 0, channel: 0; Intensity(min,max): (0 0)
Plane: 49, timepoint: 0, channel: 0; Intensity(min,max): (0 0)
Closing pixel store

Conclusion: all looks good :).
Thanks @melissalinkert

emilroz · 2014-10-28T13:58:22Z

@melissalinkert is the master CZI file meant to be in the Companion files?

This prevents pixel data from being accidentally included on the companion file list.

melissalinkert · 2014-10-31T00:40:59Z

Commit pushed to address the companion file issue.

jburel · 2014-10-31T14:39:42Z

@emilroz: did you request QA 10319 for that PR too?

emilroz · 2014-10-31T18:30:31Z

@jburel it wasn't me. There's a new mail to ome-users maybe related. But really, I don't know anything about this data.

melissalinkert · 2014-11-01T16:55:58Z

--rebased-to #1400

emilroz · 2014-11-04T10:08:58Z

No more companion files.
Good to merge!!!!

Zeiss CZI: fix dimension calculations when expected planes are missing

Zeiss CZI: fix dimension calculations when expected planes are missing

cd26576

See #12556.

Zeiss CZI: make sure pixel buffer is cleared

269e732

This prevents blank planes from containing garbage data.

melissalinkert added the develop label Oct 24, 2014

Zeiss CZI: flag all files as being pixels files

a201513

This prevents pixel data from being accidentally included on the companion file list.

melissalinkert mentioned this pull request Nov 1, 2014

Zeiss CZI: fix dimension calculations when expected planes are missing (rebased onto dev_5_0) #1400

Merged

melissalinkert added a commit that referenced this pull request Nov 4, 2014

Merge pull request #1386 from melissalinkert/12556

d787482

Zeiss CZI: fix dimension calculations when expected planes are missing

melissalinkert merged commit d787482 into ome:develop Nov 4, 2014

sbesson added this to the 5.1.0-m2 milestone Nov 26, 2014

melissalinkert deleted the 12556 branch April 13, 2015 18:03

sbesson mentioned this pull request Aug 25, 2016

CZI Reader - Correct dimensions per series #2536

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Zeiss CZI: fix dimension calculations when expected planes are missing #1386

Zeiss CZI: fix dimension calculations when expected planes are missing #1386

melissalinkert commented Oct 23, 2014

emilroz commented Oct 24, 2014

melissalinkert commented Oct 24, 2014

emilroz commented Oct 27, 2014

emilroz commented Oct 28, 2014

melissalinkert commented Oct 31, 2014

jburel commented Oct 31, 2014

emilroz commented Oct 31, 2014

melissalinkert commented Nov 1, 2014

emilroz commented Nov 4, 2014

Zeiss CZI: fix dimension calculations when expected planes are missing #1386

Zeiss CZI: fix dimension calculations when expected planes are missing #1386

Conversation

melissalinkert commented Oct 23, 2014

emilroz commented Oct 24, 2014

melissalinkert commented Oct 24, 2014

emilroz commented Oct 27, 2014

emilroz commented Oct 28, 2014

melissalinkert commented Oct 31, 2014

jburel commented Oct 31, 2014

emilroz commented Oct 31, 2014

melissalinkert commented Nov 1, 2014

emilroz commented Nov 4, 2014