Smarter Map Redraws + Tile Download Manager

[wonder-sk]

QGIS Enhancement: Smarter Map Redraws + Tile Download Manager

Date 2020/05/24

Author Martin Dobias (@wonder-sk)

Contact wonder dot sk at gmail dot com

Maintainer @wonder-sk

Version QGIS 3.16

Summary

This proposal would like to address two separate issues with map rendering:

1- Smarter redrawing of map canvas to avoid flicker

The way how map canvas rendering work currently, is that every 250 milliseconds we take what has been drawn already in the background and show it to the user. Until the first refresh (between 0ms and 250ms) we show the most recently rendered map image, possibly transformed to the new map view location. After 250ms the user sees the first preview of the newly rendered map. This generally works fine, but often some layers take longer to fetch/render and that causes unpleasant flicker - map temporarily disappears before the newly rendered map appears again. This is generally affecting raster layers more than vector layers because rasters do not appear at all before they are fully processed (WMTS and XYZ tiled rasters are an exception to that).

2- Download manager for tiled data sources

An issue that is not so visible to users is that map rending that involves raster or vector tiles from
remote sources triggers multiple requests for tiles on server, which may get canceled soon after they start and then re-requested shortly afterwards in the subsequent map rendering job. This unwanted behavior makes the download of tiles slower (and thus map redraw slower) and also creates more load on tile servers. Therefore the second half of this proposal aims to fix this behavior.

Proposed Solution

1- Smarter redraws of map canvas

The main idea here is that while a single map layer is being rendered, instead of using a completely blank map image, we would reuse a recently rendered image of the layer (transformed accordingly if the map has been zoomed/moved/rotated). Map layer renderer class will need a way to signal when the actual rendering has started - this will allow the main map renderer job to know when it is the right time to switch from the "old" image from a previous rendering to the "new" image that is being rendered. If a simple strategy to simply switch upon first render command would not be good enough, we may try some heuristics to employ a better strategy (for example, to split image to multiple regions and mix old and new parts based on where we have already started rendering and where we have not started yet).

2- Download manager for tiled data sources

The idea is to introduce a download manager that would handle requests from all map rendering jobs and even if rendering job is prematurely canceled, tiles would continue to be loaded. New map rendering jobs would not need to re-request those tiles again. This is essentially how a download manager in a web browser works.

Download manager would provide API where the client code requests URL to be fetched and gets a QObject-based reply handle which would emit finished() signal when the request is finished. In case of parallel requests to the same URL there would be only a single network request coming out. Under normal circumstances, requests would not get canceled, so that they get fully downloaded and cached so that they can be used in the future if needed.

Internally, download manager would have its own worker thread that would handle all networking and to make sure that it can work correctly independently from whether data are requested from the main thread or from a worker thread. The manager's background thread may only get started when it is first needed and then it can be terminated after some grace period when it has not been used to fetch data.

Performance Implications

This should improve performance with tile based data.

Backwards Compatibility

N/A

Issue Tracking ID(s)

(optional)

Votes

(required)

[nyalldawson]

Huge +1 to both changes. The improved map refresh behaviour has been in demand ever since QGIS 2.2 (back when @wonder-sk first implemented responsive map renders!), and the tile download issue is putting a huge strain on the OSM servers.

[PeterPetrik]

+1 too, the flickering is also quite annoying on mesh layers where the data could easily have few GB

[nyalldawson]

(this would also make a HUGE difference for animated canvases!)

[rduivenvoorde]

Great work! This has been one of the annoyances working with WMS's or big dataset from me! Thanks!

May I show an example from my work context:

As you see, after a 'drag' the 'cloud' just disappears untill the (slow/big gpkg data set) is fully rendered. In this case if would be great if indeed the 'old' layer image has just moved to the 'new' place and later replaced with the new rendering.

Later in the movie you see I'm using the Temporal Controller. In THAT case, I'm not sure what is best?

• hiding the old image (like now) I find rather ugly, also because there is no real user-feedback that 'work is being done'
• showing the old rendering (as proposed here?) is not very accurate either, as the time slider has moved, so the rendering does not 'fit' anymore with the new time

So (I'm just brainstorming here..)

• should there be some indicator (not sure where.. in MapCanvas or in LayerManager), that there is rendering work being done on one or more layers? (I am aware there is already the blue thingie in the footer...)
• should it maybe more clear that the user is looking at a 'temporary' rendering, by for example making it more transparent, or grey or ...

Anyway: great that there is some energy in this :-)

[lucamanga]

This could be REALLY useful for QGIS Server