SVS: Incorrect pyramid levels with 6.8.0

[dgault]

Issue was raised on the forum thread https://forum.image.sc/t/problem-about-opening-some-svs-slides-in-qupath-v0-3-1-bio-formats-6-8-0/61404/15

A sample file was provided on the thread which reproduces the problem. The problem seems to be as a result of changes to the handling of macro and label images in #3733

Using the provided sample file with 6.8.0 you get:

Series count = 1
Series #0 -- :
	Resolutions = 6
		sizeX[0] = 62094
		sizeX[1] = 15523
		sizeX[2] = 3880
		sizeX[3] = 1024
		sizeX[4] = 1549
		sizeX[5] = 685
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 62094
	Height = 36729
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 256 x 256
	Thumbnail size = 685 x 530
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = false
	-----
	Plane #0 <=> Z 0, C 0, T 0

With 6.8.0 and flattened resolutions you get:

Series #0 -- Series 1:
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 62094
	Height = 36729
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 256 x 256
	Thumbnail size = 685 x 530
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = false
	-----
	Plane #0 <=> Z 0, C 0, T 0

Series #1 -- Series 2:
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 15523
	Height = 9182
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 256 x 256
	Thumbnail size = 685 x 530
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = true
	-----
	Plane #0 <=> Z 0, C 0, T 0

Series #2 -- Series 3:
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 3880
	Height = 2295
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 256 x 256
	Thumbnail size = 685 x 530
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = true
	-----
	Plane #0 <=> Z 0, C 0, T 0

Series #3 -- Series 4:
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 1024
	Height = 605
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 1024 x 605
	Thumbnail size = 685 x 530
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = true
	-----
	Plane #0 <=> Z 0, C 0, T 0

Series #4 -- Series 5:
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 1549
	Height = 580
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 1549 x 580
	Thumbnail size = 685 x 530
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = true
	-----
	Plane #0 <=> Z 0, C 0, T 0

Series #5 -- Series 6:
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 685
	Height = 530
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 685 x 530
	Thumbnail size = 685 x 530
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = true
	-----
	Plane #0 <=> Z 0, C 0, T 0

With 6.7.0 you get:

Series count = 3
Series #0 -- :
	Resolutions = 4
		sizeX[0] = 62094
		sizeX[1] = 15523
		sizeX[2] = 3880
		sizeX[3] = 1024
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 62094
	Height = 36729
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 256 x 256
	Thumbnail size = 685 x 530
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = false
	-----
	Plane #0 <=> Z 0, C 0, T 0

Series #1 -- label image:
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 685
	Height = 530
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 685 x 530
	Thumbnail size = 685 x 530
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = true
	-----
	Plane #0 <=> Z 0, C 0, T 0

Series #2 -- macro image:
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 1549
	Height = 580
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 1549 x 580
	Thumbnail size = 685 x 530
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = true
	-----
	Plane #0 <=> Z 0, C 0, T 0

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/problem-about-opening-some-svs-slides-in-qupath-v0-3-1-bio-formats-6-8-0/61404/17

[sbesson]

Thanks to @petebankhead for the investigation on the source of this issue. https://forum.image.sc/t/problem-about-opening-some-svs-slides-in-qupath-v0-3-1-bio-formats-6-8-0/61404/15 seems to indicate this impacts a subset of SVS files.

As the testing of #3733 suggests, we only have representative SVS sample files in the curated OME QA repository where label and macro are stored in the metadata and allow to identify these ancillary images. Collecting representative samples for the new PFF variant(s?) is definitely a good first step.

An issue here is that reverting #3733 would solve this bug but cause a regression in other use cases.

Solving this issue will require to improve the label/macro detection logic (https://github.com/ome/bioformats/pull/3733/files#diff-314c2ae6ccef06aa694ffa85cf5843ba9a42c62e312a561914894648d61ff09aR305) to handle the different scenarios. For complete reference, the Aperio specification document section related to label and macro images:

Optionally at the end of an SVS file there may be a slide label image, which is a low resolution picture taken of the slide’s label, and/or a macro camera image, which is a low resolution picture taken of the entire slide. The label and macro images are always stripped. If present the label image is compressed with LZW compression, and the macro image with JPEG compression. By storing the label and macro images in the same file as the high resolution slide data there is no chance of an accidental mix-up between the slide contents and the slide label information.

A few initial thoughts:

• check the ImageDescription to distinguish between different implementations - feels brittle
• check the ordering of the resolution dimensions - in the example above, it is clear that 1549 is greater than 1024 and does not correspond to the next resolution level
• check other metadata in agreement with the spec above (compression, strips/tiles)

Additionally, the label/macro detection logic could be controlled via a reader-specific option allowing the user to turn it on/off.

[petebankhead]

Thanks @dgault and @sbesson
Using tiffinfo to look at the example posted on image.sc I get

TIFF Directory at offset 0x114d39fe (290273790)
  Subfile Type: (0 = 0x0)
  Image Width: 62094 Image Length: 36729 Image Depth: 1
  Tile Width: 256 Tile Length: 256
  Bits/Sample: 8
  Compression Scheme: JPEG
  Photometric Interpretation: YCbCr
  Samples/Pixel: 3
  Planar Configuration: single image plane
  ImageDescription: Aperio Leica Biosystems GT450 v1.0.0 
62094x36729 [0,0,62094x36729] (256x256) JPEG/YCC Q=91|AppMag = 40|Date = 03/17/2021|Exposure Scale = 0.000001|Exposure Time = 8|Filtered = 3|Focus Offset = -0.500000|Gamma = 2.2|Left = 32.792793273926|MPP = 0.263852|Rack = 2|ScanScope ID = SS12042|Slide = 12|StripeWidth = 4096|Time = 13:27:46|Time Zone = GMT+0800|Top = 14.066833496094
  ICC Profile: <present>, 13113264 bytes
TIFF Directory at offset 0x121f3558 (304035160)
  Subfile Type: (0 = 0x0)
  Image Width: 1024 Image Length: 605
  Bits/Sample: 8
  Compression Scheme: JPEG
  Photometric Interpretation: YCbCr
  Samples/Pixel: 3
  Rows/Strip: 605
  Planar Configuration: single image plane
  ImageDescription: Aperio Leica Biosystems GT450 v1.0.0 
1024x605 [0,0,1024x605] (1024x605) JPEG/YCC Q=100|AppMag = 40|Date = 03/17/2021|Exposure Scale = 0.000001|Exposure Time = 8|Focus Offset = 0.000000|Left = 4.3758044242859|MPP = 21.345880|Rack = 2|ScanScope ID = SS12042|Slide = 12|StripeWidth = 4096|Time = 13:27:46|Time Zone = GMT+0800|Top = 32.792793273926
  ICC Profile: <present>, 13113264 bytes
TIFF Directory at offset 0x14082a62 (336079458)
  Subfile Type: (0 = 0x0)
  Image Width: 15523 Image Length: 9182
  Tile Width: 256 Tile Length: 256
  Bits/Sample: 8
  Compression Scheme: JPEG
  Photometric Interpretation: YCbCr
  Samples/Pixel: 3
  Planar Configuration: single image plane
  ImageDescription: Aperio Leica Biosystems GT450 v1.0.0 
15523x9182 [0,0,15523x9182] (256x256) JPEG/YCC Q=91|AppMag = 40|Date = 03/17/2021|Exposure Scale = 0.000001|Exposure Time = 8|Filtered = 3|Focus Offset = -0.500000|Gamma = 2.2|Left = 32.792793273926|MPP = 1.055409|Rack = 2|ScanScope ID = SS12042|Slide = 12|StripeWidth = 4096|Time = 13:27:46|Time Zone = GMT+0800|Top = 14.066569328308
  ICC Profile: <present>, 13113264 bytes
TIFF Directory at offset 0x14e58466 (350585958)
  Subfile Type: (0 = 0x0)
  Image Width: 3880 Image Length: 2295
  Tile Width: 256 Tile Length: 256
  Bits/Sample: 8
  Compression Scheme: JPEG
  Photometric Interpretation: YCbCr
  Samples/Pixel: 3
  Planar Configuration: single image plane
  ImageDescription: Aperio Leica Biosystems GT450 v1.0.0 
3880x2295 [0,0,3880x2295] (256x256) JPEG/YCC Q=91|AppMag = 40|Date = 03/17/2021|Exposure Scale = 0.000001|Exposure Time = 8|Filtered = 3|Focus Offset = -0.500000|Gamma = 2.2|Left = 32.792793273926|MPP = 4.221636|Rack = 2|ScanScope ID = SS12042|Slide = 12|StripeWidth = 4096|Time = 13:27:46|Time Zone = GMT+0800|Top = 14.064458847046
  ICC Profile: <present>, 13113264 bytes
TIFF Directory at offset 0x15b6e470 (364307568)
  Subfile Type: reduced-resolution image (1 = 0x1)
  Image Width: 685 Image Length: 530
  Bits/Sample: 8
  Compression Scheme: LZW
  Photometric Interpretation: RGB color
  Samples/Pixel: 3
  Rows/Strip: 530
  Planar Configuration: single image plane
  Predictor: horizontal differencing 2 (0x2)
TIFF Directory at offset 0x15b899a0 (364419488)
  Subfile Type: reduced-resolution image (9 = 0x9)
  Image Width: 1549 Image Length: 580
  Bits/Sample: 8
  Compression Scheme: JPEG
  Photometric Interpretation: RGB color
  Samples/Pixel: 3
  Rows/Strip: 580
  Planar Configuration: single image plane

The subfile type looks potentially useful, as does the (lack of) ImageDescription. Although I am faintly aware that some other software claims to write svs (e.g. 3DHistech's Slide Converter) and I don't know how closely it follows the Aperio specification document.

In case QuPath could be useful for testing, I've just merged a PR that makes it easier to switch between Bio-Formats versions. The application can be launched from the source directory with

./gradlew run -Pbioformats-version=6.8.0

or

./gradlew run -Pbioformats-version=6.7.0

The latest code uses the trick of eliminating the 'impossible' resolutions, i.e. where the downsample value decreases at the next level. However this was still not enough to get things to work in 6.8.0: it seems that the levels that were requested were incorrect.

In the meantime, I plan to make a QuPath release that reverts to 6.7.0 soon.

[dgault]

Reopening this as there is still an ongoing issue in the thread which will require some further investigation: https://forum.image.sc/t/problem-about-opening-some-svs-slides-in-qupath-v0-3-1-bio-formats-6-8-0/61404/26

The file in question is located at https://dbarchive.biosciencedbc.jp/data/open-tggates-pathological-images/LATEST/images/2-nitrofluorene/Kidney/65141.svs

[sbesson]

Cross-linking to Pete's answer on the thread, the way OpenSlide makes the assignment between WSI and label/macro for the sample file mentioned in #3757 (comment) is based on the usage of tiles vs strips. This is in agreement with the relevant line of the specification - see #3757 (comment) and might be the next thing to try out in the reader

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/fiji-wont-recognize-svs-files/63528/2

[sbesson]

Possibly instance of this issue was reported in https://forum.image.sc/t/preview-error-when-tcga-svs-image-was-imported/67945 using a sample SVS file from the TCGA repository.

Loooking at the TIFF structure

TIFF Directory at offset 0x3650c6ee (911263470)
  Subfile Type: (0 = 0x0)
  Image Width: 95616 Image Length: 57015 Image Depth: 1
  Tile Width: 240 Tile Length: 240
  Bits/Sample: 8
  Compression Scheme: JPEG
  Photometric Interpretation: RGB color
  YCbCr Subsampling: 2, 2
  Samples/Pixel: 3
  Planar Configuration: single image plane
  ImageDescription: Aperio Image Library v12.0.15 
97536x57115 [0,100 95616x57015] (240x240) JPEG/RGB Q=30|AppMag = 40|StripeWidth = 2032|ScanScope ID = SS1763CNTLR|Filename = TCGA-OR-A5K9-01Z-00-DX2|Title = TCGA-OR-A5K9-01Z-00-DX2|Date = 07/15/14|Time = 23:07:20|Time Zone = GMT-04:00|User = 73161a05-c878-4a2b-ad59-cfa0145e6534|Parmset = GOG136 on O: Drive|MPP = 0.2525|Left = 23.605759|Top = 22.683418|LineCameraSkew = 0.000000|LineAreaXOffset = 0.006121|LineAreaYOffset = -0.001548|Focus Offset = 0.000000|DSR ID = resc3-dsr1|ImageID = 159473|Exposure Time = 109|Exposure Scale = 0.000001|DisplayColor = 0|OriginalWidth = 97536|OriginalHeight = 57115|ICC Profile = ScanScope v1
  ICC Profile: <present>, 141992 bytes
  JPEG Tables: (574 bytes)
TIFF Directory at offset 0x3661939e (912364446)
  Subfile Type: (0 = 0x0)
  Image Width: 1024 Image Length: 610 Image Depth: 1
  Bits/Sample: 8
  Compression Scheme: JPEG
  Photometric Interpretation: RGB color
  YCbCr Subsampling: 2, 2
  Samples/Pixel: 3
  Rows/Strip: 16
  Planar Configuration: single image plane
  ImageDescription: Aperio Image Library v12.0.15 
95616x57015 -> 1024x610 - |AppMag = 40|StripeWidth = 2032|ScanScope ID = SS1763CNTLR|Filename = TCGA-OR-A5K9-01Z-00-DX2|Title = TCGA-OR-A5K9-01Z-00-DX2|Date = 07/15/14|Time = 23:07:20|Time Zone = GMT-04:00|User = 73161a05-c878-4a2b-ad59-cfa0145e6534|Parmset = GOG136 on O: Drive|MPP = 0.2525|Left = 23.605759|Top = 22.683418|LineCameraSkew = 0.000000|LineAreaXOffset = 0.006121|LineAreaYOffset = -0.001548|Focus Offset = 0.000000|DSR ID = resc3-dsr1|ImageID = 159473|Exposure Time = 109|Exposure Scale = 0.000001|DisplayColor = 0|OriginalWidth = 97536|OriginalHeight = 57115|ICC Profile = ScanScope v1
  JPEG Tables: (289 bytes)
TIFF Directory at offset 0x3e5e7e46 (1046380102)
  Subfile Type: (0 = 0x0)
  Image Width: 23904 Image Length: 14253 Image Depth: 1
  Tile Width: 240 Tile Length: 240
  Bits/Sample: 8
  Compression Scheme: JPEG
  Photometric Interpretation: RGB color
  YCbCr Subsampling: 2, 2
  Samples/Pixel: 3
  Planar Configuration: single image plane
  ImageDescription: Aperio Image Library v12.0.15 
97536x57115 [0,100 95616x57015] (240x240) -> 23904x14253 JPEG/RGB Q=65
  JPEG Tables: (574 bytes)
TIFF Directory at offset 0x3f2b6e64 (1059810916)
  Subfile Type: (0 = 0x0)
  Image Width: 5976 Image Length: 3563 Image Depth: 1
  Tile Width: 240 Tile Length: 240
  Bits/Sample: 8
  Compression Scheme: JPEG
  Photometric Interpretation: RGB color
  YCbCr Subsampling: 2, 2
  Samples/Pixel: 3
  Planar Configuration: single image plane
  ImageDescription: Aperio Image Library v12.0.15 
97536x57115 [0,100 95616x57015] (240x240) -> 5976x3563 JPEG/RGB Q=82
  JPEG Tables: (574 bytes)
TIFF Directory at offset 0x3f710cb2 (1064373426)
  Subfile Type: (0 = 0x0)
  Image Width: 2988 Image Length: 1781 Image Depth: 1
  Tile Width: 240 Tile Length: 240
  Bits/Sample: 8
  Compression Scheme: JPEG
  Photometric Interpretation: RGB color
  YCbCr Subsampling: 2, 2
  Samples/Pixel: 3
  Planar Configuration: single image plane
  ImageDescription: Aperio Image Library v12.0.15 
97536x57115 [0,100 95616x57015] (240x240) -> 2988x1781 JPEG/RGB Q=91
  JPEG Tables: (574 bytes)

With the current version of Bio-Formats, this is detected as a single multi-resolution image:

Checking file format [Aperio SVS]
Initializing reader
SVSReader initializing /uod/idr/scratch/inbox/TCGA-OR-A5K9-01Z-00-DX2.7C0DDEF6-8CCC-4640-820F-7703616E2FEF.svs
Reading IFDs
Populating metadata
Populating OME metadata
Initialization took 0.325s

Reading core metadata
filename = /uod/idr/scratch/inbox/TCGA-OR-A5K9-01Z-00-DX2.7C0DDEF6-8CCC-4640-820F-7703616E2FEF.svs
Series count = 1
Series #0 :
	Resolutions = 5
		sizeX[0] = 95616
		sizeX[1] = 23904
		sizeX[2] = 5976
		sizeX[3] = 2988
		sizeX[4] = 1024
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 95616
	Height = 57015
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 240 x 240
	Thumbnail size = 128 x 76
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = false

but as highlighted by the original thread, the navigation between the pyramidal levels shows clear inconsistency which points either to an ordering issue of to the fact some of the IFDs should be treated as separate images.

Coming back to the statement above and in agreement with the Aperio specification document, relying on whether IFDs are written using strips and uses a JPEG compression seems like a consistent way to identify these intermediate planes.

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/preview-error-when-tcga-svs-image-was-imported/67945/2

[sbesson]

Retested briefly the sample file mentioned in #3757 (comment) with an older version of Bio-Formats (prior to the changes introduced in #3733).

In Bio-Formats 6.7.0, the data representation was as follows i.e. 3 series one with pyramidal levels and two ancillary images

bash-4.2$ ./bftools/showinf -nopix -noflat -nometa /uod/idr/scratch/inbox/TCGA-OR-A5K9-01Z-00-DX2.7C0DDEF6-8CCC-4640-820F-7703616E2FEF.svs 
Checking file format [Aperio SVS]
Initializing reader
SVSReader initializing /uod/idr/scratch/inbox/TCGA-OR-A5K9-01Z-00-DX2.7C0DDEF6-8CCC-4640-820F-7703616E2FEF.svs
Reading IFDs
Populating metadata
Populating OME metadata
Initialization took 0.287s

Reading core metadata
filename = /uod/idr/scratch/inbox/TCGA-OR-A5K9-01Z-00-DX2.7C0DDEF6-8CCC-4640-820F-7703616E2FEF.svs
Series count = 3
Series #0 :
	Resolutions = 3
		sizeX[0] = 95616
		sizeX[1] = 23904
		sizeX[2] = 1024
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 95616
	Height = 57015
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 240 x 240
	Thumbnail size = 128 x 76
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = false
Series #1 :
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 5976
	Height = 3563
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 240 x 240
	Thumbnail size = 128 x 76
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = true
Series #2 :
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 2988
	Height = 1781
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 240 x 240
	Thumbnail size = 128 x 76
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = true
bash-4.2$ ./bftools/showinf -nopix -noflat -nometa /uod/idr/scratch/inbox/TCGA-OR-A5K9-01Z-00-DX2.7C0DDEF6-8CCC-4640-820F-7703616E2FEF.svs 
Checking file format [Aperio SVS]
Initializing reader
SVSReader initializing /uod/idr/scratch/inbox/TCGA-OR-A5K9-01Z-00-DX2.7C0DDEF6-8CCC-4640-820F-7703616E2FEF.svs
Reading IFDs
Populating metadata
Populating OME metadata
Initialization took 0.287s

Reading core metadata
filename = /uod/idr/scratch/inbox/TCGA-OR-A5K9-01Z-00-DX2.7C0DDEF6-8CCC-4640-820F-7703616E2FEF.svs
Series count = 3
Series #0 :
	Resolutions = 3
		sizeX[0] = 95616
		sizeX[1] = 23904
		sizeX[2] = 1024
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 95616
	Height = 57015
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 240 x 240
	Thumbnail size = 128 x 76
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = false
Series #1 :
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 5976
	Height = 3563
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 240 x 240
	Thumbnail size = 128 x 76
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = true
Series #2 :
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 2988
	Height = 1781
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 240 x 240
	Thumbnail size = 128 x 76
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = true

As demonstrated in #3757 (comment), Bio-Formats 6.10.0, creates an internal representation of this image as a single multi-resolution image with 5 pyramidal levels. This seems definitely more in-line with the internal structure of the file.

Importing the file into an OMERO.server 5.6.3 (including Bio-Formats 5.6.1) leads to the file being imported as three images. The second one internally generates a pyramid (as it exceeds the maximum plane dimensions). The first image is usable but the user experience is sub-optimal and reveals the absence of intermediate resolutions:

Importing the file into an development OMERO.server (including the HEAD of Bio-Formats) leads to the file being imported as a single image with no pyramid generation. The thumbnail is broken and the data displayed at low levels of resolution are incorrect as shown in https://forum.image.sc/t/preview-error-when-tcga-svs-image-was-imported/67945

At high-level of resolutions, using the viewport URL to compare equivalent regions on both systems seems to indicate that the data is identical

For this particular sample file, my overall feeling is that

• the representation of this data as 3 separate images was incorrect prior to Bio-Formats 5.7.0. The first image included a minimal set of resolutions levels making it usable by clients like QuPath or even OMERO but the import was leading to extra images and unnecesary pyramid generation
• the change in Bio-Formats 6.8.0 (possibly including the fix for 6.8.1?) fixed the data representation issue resulting in a single multi-resolution image with additional pyramidal levels. However this resulted in a secondary issue associated with pyramidal levels smaller than the highest resolution which leads to the rendering errors described in the various image.sc forum posts and in the issue above.

The description of #3764 includes a reference to IFD re-ordering which might be related so possibly we want to reinclude this PR and test it against the two sample files @dgault ?

Also quoting another part of the Aperio specification

The first image in an SVS file is always the baseline image (full resolution). This image is always tiled, usually with a tile size of 240 x 240 pixels. The second image is always a thumbnail, typically with dimensions of about 1024 x 768 pixels. Unlike the other slide images, the thumbnail image is always stripped. Following the thumbnail there may be one or more intermediate “pyramid” images. These are always compressed with the same type of compression as the baseline image, and have a tiled organization with the same tile size.

So the expected IFD order seems to be

• highest resolution level (stored in tiles)
• thumbnail aka lowest resolution (stored in strips)
• (optional) intermediate resolution levels (stored in tiles)
• (optional) label image (stored in strips, LZW compressed)
• (optional) macro image (stored in strips, JPEG compressed)

This seems to be consistent with the two output of tiffinfo pasted in #3757 (comment) and #3757 (comment)

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/correct-way-to-convert-svs-slides-for-omero-import/50942/28

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/correct-way-to-convert-svs-slides-for-omero-import/50942/29

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/correct-way-to-convert-svs-slides-for-omero-import/50942/30

[melissalinkert]

Looked at this again in the context of #4081. Based on that work and #3757 (comment), it looks the problem at this point is down to the thumbnail image only, and specifically the fact that the thumbnail's aspect ratio differs from the other resolutions and the scale factor between the thumbnail and next-largest resolution is different from scale factors between other pairs of adjacent resolutions in the pyramid.

A few possible ways forward:

1. Keep using the thumbnail as the smallest resolution in the pyramid. I think this is reasonably consistent with how ImageScope represents the data, but obviously causes problems for OMERO and other applications that assume constant scale factors as described above.
2. Throw away the thumbnail entirely. Arguably a small amount of data loss, but should improve the OMERO use case with minimal effort.
3. Split the thumbnail into a separate series outside of the pyramid. This preserves the thumbnail image and should fix pyramid viewing in OMERO, but requires more effort and has higher impact than (1) and (2). Impact could be slightly lessened by making sure the thumbnail is the very last series; I think that would mean that existing OMERO imports only need to be re-imported if the separate thumbnail image is desired.
4. Any combination of 1, 2, and 3 configurable with an option.

My personal preference is (2) by default, maybe with an option to switch to (1).

[sbesson]

Coming back late (apologies) to this issue, my preference would also go to implement proposal (2) with a reader option to fall back to the current behavior.

The sub-resolution API does not impose many requirements on the relationship between the consecutive resolutions so none of the options discussed above is fundamentally incorrect.

Taking the most extreme layout, a SVS file can be minimally composed of 2 IFDs: the high-resolution and the thumbnail. Without flattening the resolutions, the current SVS reader API would detect such a file as a single image with 2 resolution levels. This representation would be highly unusable for resolution-aware clients which would typically expect pyramidal levels to be downsampled homogeneously alongside each dimension using a predetermiend scale factor (usually 2, 3, 4).
Instead, (2) feels like a much more faithful representation of the nature of the data i.e. effectively a single-resolution whole slide image.

To retain the ability to expose the thumbnail IFD and offer some form of backwards compatibility for clients who might be relying on the current behavior, adding a reader-specific option would be fine with me. Alternatively, it might be possible to expose this thumbnail using the SVSReader.openThumbBytes() API.

[melissalinkert]

All makes sense. #4081 needs to be merged first, but then it probably makes sense to do this for 7.1.0 or 7.2.0 so we can draw a line under major SVS issues. I've added to the 7.1.0 milestone so we don't forget, but can push to 7.2.0 depending on timelines.

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/problem-about-opening-some-svs-slides-in-qupath-v0-3-1-bio-formats-6-8-0/61404/29