Some source resolutions are sometimes skipped leading to reading too much tiles

[jfbourgon]

Looking at the log entry we noticed that some source resolutions are sometimes skipped leading to reading too much tiles. In our test case both client and source data are requested/stored according to Web Mercator reference system so we were expecting to use all the available resolutions as we zoom in. Here is an example where we skipped level 11 even if the evaluated target resolution is closed to level 11 resolution @ merc-CS TO CellSize(76.43702828517661,76.43702828517416), the level 12 is used CellSize(38.21851414258813,38.21851414258813)) and 112 tiles are read.

Log output example:

20:57:58.808 [scala-execution-context-global-63] TRACE geotrellis.server.LayerExtent - [LayerExtent] Retrieved parameters for extent (Extent(-1.2869320267233701E7, 8831152.06292784, -1.2723784165378725E7, 8892301.68555598)) and cellsize (CellSize(76.43702828517661,76.43702828517416)): Map(2015-nir -> SimpleOgcLayer(2015-ndvi,NDVI of temporal serie LandSat,EPSG:3857,GeoTrellisRasterSourceLegacy(s3://datacube-dev-data/landsat/mosaic, Layer(name = "2015-nir", zoom = 12)),Some(ColorMapStyle(official,2015 NDVI blue/green colors map,geotrellis.raster.render.DoubleColorMap@67dba24b,List(LegendModel(image/png,36,36,OnlineResourceModel(simple,http://dy0a51yzxbhxw.cloudfront.net/images/canada-icon.png,None,None,None,None)))))), 2015-red -> SimpleOgcLayer(2015-ndvi,NDVI of temporal serie LandSat,EPSG:3857,GeoTrellisRasterSourceLegacy(s3://datacube-dev-data/landsat/mosaic, Layer(name = "2015-red", zoom = 12)),Some(ColorMapStyle(official,2015 NDVI blue/green colors map,geotrellis.raster.render.DoubleColorMap@67dba24b,List(LegendModel(image/png,36,36,OnlineResourceModel(simple,http://dy0a51yzxbhxw.cloudfront.net/images/canada-icon.png,None,None,None,None)))))))
20:57:58.809 [scala-execution-context-global-66] DEBUG g.store.GeoTrellisResampleRasterSourceLegacy - read(Extent(-1.2869320267233701E7, 8831152.06292784, -1.2723784165378725E7, 8892301.68555598)) = s3://datacube-dev-data/landsat/mosaic Layer(name = "2015-red", zoom = 12) CellSize(38.21851414258813,38.21851414258813) @ merc-CS TO CellSize(76.43702828517661,76.43702828517416) -- reading 112 tiles
20:57:58.811 [scala-execution-context-global-63] DEBUG g.store.GeoTrellisResampleRasterSourceLegacy - read(Extent(-1.2869320267233701E7, 8831152.06292784, -1.2723784165378725E7, 8892301.68555598)) = s3://datacube-dev-data/landsat/mosaic Layer(name = "2015-nir", zoom = 12) CellSize(38.21851414258813,38.21851414258813) @ merc-CS TO CellSize(76.43702828517661,76.43702828517416) -- reading 112 tiles

[pomadchin]

This task would be to update up to the latest GeoTrellis version that contains locationtech/geotrellis#3196 and to expose overview strategy that would be used for the layer (this task is similar to #229).

[pomadchin]

Moved this into Done; awaiting @jfbourgon to confirm the state of the issue; I'm really wondering would #252 work or not.

This message is left here for the history, otherwise we can forget why this is not closed / how the other issues are related to it.

[jfbourgon]

I'm glad to confirm that the new Overview Strategy implementation works like a charm. Please see below for evidence from the log output:

With Default settings (equivalent to "auto-higher-resolution"), the zoom level 11 was skipped and number of tiles read was varying

Zoom 10 GetMap request

20:03:08.534 [blaze-selector-1] WARN  org.http4s.server.blaze.BlazeServerBuilder - HTTP/2 support requires TLS. Falling back to HTTP/1.
20:03:08.775 [scala-execution-context-global-31] DEBUG geotrellis.store.GeoTrellisResampleRasterSource - read(Extent(-1.3161691900424499E7, 8964534.677285472, -1.297105795188127E7, 9086833.922541756)) = s3://datacube-dev-data/landsat Layer(name = "landcover2", zoom = 10) CellSize(152.8740565703525,152.8740565703525) @ merc-CS TO CellSize(152.8740565703514,152.87405657035532) -- reading 24 tiles
20:03:08.778 [scala-execution-context-global-28] DEBUG geotrellis.store.GeoTrellisResampleRasterSource - read(Extent(-1.4752613988638015E7, 7646263.468965178, -1.21935787186786E7, 1.023713476120537E7)) = s3://datacube-dev-data/landsat Layer(name = "landcover2", zoom = 0) CellSize(156543.03392804097,156543.03392804097) @ merc-CS TO CellSize(156543.03392804097,156543.03392804097) -- reading 1 tiles

Zoom 11 GetMap request

20:03:10.684 [blaze-selector-2] WARN  org.http4s.server.blaze.BlazeServerBuilder - HTTP/2 support requires TLS. Falling back to HTTP/1.
20:03:10.941 [scala-execution-context-global-28] DEBUG geotrellis.store.GeoTrellisResampleRasterSource - read(Extent(-1.3104287692182332E7, 8992663.503694419, -1.3008970717910718E7, 9053813.12632256)) = s3://datacube-dev-data/landsat Layer(name = "landcover2", zoom = 12) CellSize(38.21851414258813,38.21851414258813) @ merc-CS TO CellSize(76.4370282851757,76.4370282851765) -- reading 77 tiles
20:03:10.951 [scala-execution-context-global-31] DEBUG geotrellis.store.GeoTrellisResampleRasterSource - read(Extent(-1.4752613988638015E7, 7646263.468965178, -1.21935787186786E7, 1.023713476120537E7)) = s3://datacube-dev-data/landsat Layer(name = "landcover2", zoom = 0) CellSize(156543.03392804097,156543.03392804097) @ merc-CS TO CellSize(156543.03392804097,156543.03392804097) -- reading 1 tiles

Zoom 12 GetMap request

20:03:15.304 [blaze-selector-3] WARN  org.http4s.server.blaze.BlazeServerBuilder - HTTP/2 support requires TLS. Falling back to HTTP/1.
20:03:15.500 [scala-execution-context-global-33] DEBUG geotrellis.store.GeoTrellisResampleRasterSource - read(Extent(-1.3075585588061249E7, 9006727.91689889, -1.3027927100925442E7, 9037302.728212966)) = s3://datacube-dev-data/landsat Layer(name = "landcover2", zoom = 12) CellSize(38.21851414258813,38.21851414258813) @ merc-CS TO CellSize(38.21851414258785,38.21851414259523) -- reading 24 tiles
20:03:15.503 [scala-execution-context-global-31] DEBUG geotrellis.store.GeoTrellisResampleRasterSource - read(Extent(-1.4752613988638015E7, 7646263.468965178, -1.21935787186786E7, 1.023713476120537E7)) = s3://datacube-dev-data/landsat Layer(name = "landcover2", zoom = 0) CellSize(156543.03392804097,156543.03392804097) @ merc-CS TO CellSize(156543.03392804097,156543.03392804097) -- reading 1 tiles

With overview-strategy = "auto-0" settings, the zoom level 11 is no longer skipped and the number of tiles read is consistent

Zoom 10 GetMap request

20:10:55.785 [blaze-selector-0] WARN  org.http4s.server.blaze.BlazeServerBuilder - HTTP/2 support requires TLS. Falling back to HTTP/1.
20:10:56.045 [scala-execution-context-global-29] DEBUG geotrellis.store.GeoTrellisResampleRasterSource - read(Extent(-1.3081127272611925E7, 8777722.5801565, -1.2890493324068695E7, 8900021.825412782)) = s3://datacube-dev-data/landsat Layer(name = "landcover2", zoom = 10) CellSize(152.8740565703525,152.8740565703525) @ merc-CS TO CellSize(152.8740565703529,152.874056570353) -- reading 24 tiles
20:10:56.051 [scala-execution-context-global-27] DEBUG geotrellis.store.GeoTrellisResampleRasterSource - read(Extent(-1.4752613988638015E7, 7646263.468965178, -1.21935787186786E7, 1.023713476120537E7)) = s3://datacube-dev-data/landsat Layer(name = "landcover2", zoom = 0) CellSize(156543.03392804097,156543.03392804097) @ merc-CS TO CellSize(156543.03392804097,156543.03392804097) -- reading 1 tiles

Zoom 11 GetMap request

20:10:57.475 [blaze-selector-1] WARN  org.http4s.server.blaze.BlazeServerBuilder - HTTP/2 support requires TLS. Falling back to HTTP/1.
20:10:57.698 [scala-execution-context-global-27] DEBUG geotrellis.store.GeoTrellisResampleRasterSource - read(Extent(-1.4752613988638015E7, 7646263.468965178, -1.21935787186786E7, 1.023713476120537E7)) = s3://datacube-dev-data/landsat Layer(name = "landcover2", zoom = 0) CellSize(156543.03392804097,156543.03392804097) @ merc-CS TO CellSize(156543.03392804097,156543.03392804097) -- reading 1 tiles
20:10:57.700 [scala-execution-context-global-29] DEBUG geotrellis.store.GeoTrellisResampleRasterSource - read(Extent(-1.301875465753122E7, 8800500.814585486, -1.2923437683259606E7, 8861650.437213626)) = s3://datacube-dev-data/landsat Layer(name = "landcover2", zoom = 11) CellSize(76.43702828517625,76.43702828517625) @ merc-CS TO CellSize(76.4370282851757,76.43702828517416) -- reading 24 tiles

Zoom 12 GetMap request

20:11:01.299 [blaze-selector-2] WARN  org.http4s.server.blaze.BlazeServerBuilder - HTTP/2 support requires TLS. Falling back to HTTP/1.
20:11:01.507 [scala-execution-context-global-31] DEBUG geotrellis.store.GeoTrellisResampleRasterSource - read(Extent(-1.2987568349990869E7, 8811889.931799974, -1.2939909862855062E7, 8842464.743114045)) = s3://datacube-dev-data/landsat Layer(name = "landcover2", zoom = 12) CellSize(38.21851414258813,38.21851414258813) @ merc-CS TO CellSize(38.21851414258785,38.21851414258825) -- reading 24 tiles
20:11:01.512 [scala-execution-context-global-29] DEBUG geotrellis.store.GeoTrellisResampleRasterSource - read(Extent(-1.4752613988638015E7, 7646263.468965178, -1.21935787186786E7, 1.023713476120537E7)) = s3://datacube-dev-data/landsat Layer(name = "landcover2", zoom = 0) CellSize(156543.03392804097,156543.03392804097) @ merc-CS TO CellSize(156543.03392804097,156543.03392804097) -- reading 1 tiles