Request of tile sizes that are not part of the scaleFactors

[jodogne]

Hello,

Thanks for your great work on OSD! I face an issue with a very basic tiled image, whose info.json reads as follows:

{
   "@context" : "http://iiif.io/api/image/2/context.json",
   "@id" : "http://localhost:8042/wsi/iiif/tiles/5b29b598-98f24142-200eb464-db39ed07-8d44f0a3",
   "height" : 3000,
   "profile" : "http://iiif.io/api/image/2/level0.json",
   "protocol" : "http://iiif.io/api/image",
   "sizes" : [
      {
         "height" : 750,
         "width" : 1250
      },
      {
         "height" : 3000,
         "width" : 5000
      }
   ],
   "tiles" : [
      {
         "height" : 512,
         "scaleFactors" : [ 4, 1 ],
         "width" : 512
      }
   ],
   "width" : 5000
}

As can be seen, this image offers two tile sizes: 2048x2048 and 512x512, expressed in coordinates of the original image. However, if I zoom on the top-left part of the image using OSD, I observe that the browser is making requests such as:

http://localhost:8042/wsi/iiif/tiles/5b29b598-98f24142-200eb464-db39ed07-8d44f0a3/0,1024,1024,1024/512,/0/default.jpg

This is surprising to me, as this seems to correspond to a 1024x1024 tile, which is not part of my scaleFactors, so my server answers with a 404 error. I have observed this behavior both with both OSD 3.1 and OSD 4.0. On OSD 4.1, things can go worse, as it sometimes emits requests with negative sizes when I zoom on the bottom-right part of the image:

http://localhost:8042/wsi/iiif/tiles/5b29b598-98f24142-200eb464-db39ed07-8d44f0a3/4096,5120,904,-2120/512,/0/default.jpg

Is this a consequence of a bad use of OSD on my side? Thanks in advance for your support!

Kind Regards,
Sébastien-

[jodogne]

Small update: This issue is not specific to Image API 2.1.1. It can also be reproduced if using Image API 3.0.0, with the following image.json:

{
   "@context" : "http://iiif.io/api/image/3/context.json",
   "height" : 3000,
   "id" : "http://localhost:8042/wsi/iiif/tiles/21bc8b87-07ec0228-597e2002-2710523a-d96927e9",
   "profile" : "level0",
   "protocol" : "http://iiif.io/api/image",
   "sizes" : [
      {
         "height" : 750,
         "width" : 1250
      },
      {
         "height" : 3000,
         "width" : 5000
      }
   ],
   "tiles" : [
      {
         "height" : 512,
         "scaleFactors" : [ 4, 1 ],
         "width" : 512
      }
   ],
   "type" : "ImageService3",
   "width" : 5000
}

[iangilman]

I'm not so familiar with IIIF. @glenrobson @azaroth42 @ruven @ahankinson @tomcrane Any thoughts?

[ahankinson]

@jodogne what image server are you using?

[jodogne]

@ahankinson This is Orthanc, which has just released IIIF support for whole-slide imaging (telepathology) and teleradiology.

If this is of interest to you, here the link to a URL to an IIIF manifest that is dynamically generated by our demo server: https://wsi.orthanc-server.com/orthanc/wsi/iiif/series/645ce678-fc6c0544-2ce7fcc0-0c3f2ba6-2e4b63b5/manifest.json

This manifest can notably opened as a resource by the online demo of Mirador, which internally uses OpenSeadragon. Note that the tiles are generated on-the-fly from DICOM images.

[ahankinson]

Running that image through the IIIF Image API validator throws a bunch of errors:

https://iiif.io/api/image/validator/

Notably, this image request asking for an image 614 x 652 returns an image that is 1348 x 938:

https://wsi.orthanc-server.com/orthanc/wsi/iiif/tiles/645ce678-fc6c0544-2ce7fcc0-0c3f2ba6-2e4b63b5/full/614,652/0/default.jpg

So maybe something is wrong with the server itself?

[jodogne]

Running that image through the IIIF Image API validator throws a bunch of errors:

Thanks for your investigation! As this is the first release of IIIF on our side, we were expecting some discrepancies with respect to the IIIF standard. We'll have a look at the validator.

That being said, the original question is not related to Orthanc, and I feel like there is also some investigation to be done in OSD.

[glenrobson]

Hi @jodogne, great to see DICOM images with IIIF!

Have you changed the image server available at: https://wsi.orthanc-server.com/orthanc/wsi/iiif/tiles/? I've tested it in OSD 3.1.1 and OSD 4.0.0 and it looks to be working great now.

From your description in the first comment it seems that OSD was assuming a scaleFactor of 2 in the list of scaleFactors even though you hadn't specified it in the list. I can see from the orthanc-server.com you now have the following scaleFactors:

scaleFactors": [
            1,
            2,
            4,
            8,
            16,
            32
]

which is a more common structure including the powers of 2 without any gaps. I suspect this has solved the issue you were having with OSD but let me know if you do need scaleFactors of 4 and 1 only and I can do some testing.

[jodogne]

Hi @glenrobson, thanks for your positive feedback, and sorry for the delay (I was on family vacations)!
You are right about the fact that I had to implement workarounds to truncate scaleFactors that don't follow the sequence of the powers of 2. This is the configuration option ForcePowersOfTwoScaleFactors that is explained in the documentation of the whole-slide imaging plugin for Orthanc.
Unfortunately, many whole-slide scanners will natively generate multi-resolution images that don't follow this sequence, with gaps in the zoom levels. In such a situation, the compatibility mode of the Orthanc plugin will drop the lower-resolution levels even if they are present, which forces the loading of many tiles even though a coarser-resolution tiles are available. And even though the Orthanc DICOM-izer is able to recreate a "regular" pyramid, this is a time-consuming operation. So, I think that it still makes sense for OSD to try and support such gaps.
NB: Also, besides the gaps, note that whole-slide scanners will typically generate sequences that are not powers of 2.

[iangilman]

Are gaps like these supported in the IIIF standard but not OSD? Or would it be a matter of updating both?

[jodogne]

Are gaps like these supported in the IIIF standard but not OSD? Or would it be a matter of updating both?

From my understanding, nothing in the IIIF Image API 3.0 standard prevents gaps from occurring in scaleFactors. The only restriction is that scale factors must be positive integers, which implies that gaps can occur, and that scale factors are not necessarily powers of 2.

[ahankinson]

I think it's a gap in the spec, only because a search of the API issues doesn't return any discussion about gaps in the scale factors. Reading between the lines, I think this probably means that it wasn't discussed by the editors. (There are other discussions about non-integer scale factors, non-power-of-two scale factors, and the like).

Would be good to add an issue to clarify this in the spec.

[azaroth42]

It was discussed and decided to have any set of integers, not just ordered factors of two. So it's intentionally perfectly valid to have an arbitrary set of positive integers for supported scale factors.

[iangilman]

@azaroth42 Good to know!

So yeah, it seems it would be good for OSD to support any kind of scale factor that's valid in IIIF. I'm actually not sure what would be involved in that, but if somebody's interested in pursuing it I can help figure it out.

[ahankinson]

The Image API Specs say:

The set of resolution scaling factors for the image’s predefined tiles, expressed as positive integers by which to divide the full size of the image. For example, a scale factor of 4 indicates that the service can efficiently deliver images at 1/4 or 25% of the height and width of the full image. A particular scale factor value should appear only once in the tiles array.

What's not clear to me from this is whether the given scale factors are required to be provided, or only hints at efficient delivery. In other words, if you have scale factors of 1, 2, and 8, does it mean that scale factor 4 is completely unavailable, and that the server will refuse to serve it, or it's just not a pre-defined scale factor?

It's "...the service can efficiently deliver images..." that's hanging me up; It doesn't say "...the service must deliver images..."

[azaroth42]

It depends on the conformance level of the server. A level 0 won't deliver anything as the SF4 tiles wouldn't exist, but a level 2 will generate them as needed, just not as efficiently. So we were intentionally vague to allow both implementations to conform to the specification. See: https://iiif.io/api/image/3.0/compliance/#32-size