Tile creation and update

[Jeremy-Gaillard]

Not ready for pull yet. Pull request for visibility, since quite a lot of things are changing.
Solve (at least partially) issues #70, #54 and #57

Content:

• Tile instantiation pushed from command to quadtree. The tile are created empty, save from the metadata (tile coordinates, bounding box, ...)
• Commands feed data to a tile. It is recommended that a command's requester is the tile that needs the data
• New update mechanism (see below)

How to handle tile loading:

• A layer only has one provider. If you need multiple provider for a tile, build a composite provider such as TileProvider.
• Tiles have an getStatus() function which returns the type of data it currently needs in the form of an array of string. "none" or undefined indicate that no data is needed, "ready" indicates the tile is ready to be rendered. Any string will result in the creation of a new command, an array of string will result in the creation of multiple commands. getStatus() should return only the types of the data that it can currently load (e.g. don't return "elevation" if you don't have the underlying geometry)
• The provider linked to the layer must have an executeCommand(command) function. The command object has four attributes: type (the type of data needed), requester (the tile that needs data), layer (the layer of the tile) and parameters (any other parameters you may need)
• To each layer is bound a NodeProcess. NodeProcess decides whether a node should be displayed, refined, etc. NodeProcess.js may need tweaking if you want to reuse it for your own data structure. It should work for any hierarchical structure on a globe, with a refinement strategy which replaces parent tile with children tiles (i.e. no additive refinement).

Important tile attributes:

• cullable: must be set to true as soon as the tile can be tested for culling. Often, this will mean the bounding box is ready
• pending: automatically set to true when a command is created. Must be set to false when the expected data is added to the tile Handled by the interface commander
• visible: the tile is visible but not necessarily rendered
• material.visible: the tile is rendered
• disposed: set to true when tile is being deleted
• divided: the tile has been divided (refined). Set to true by the Quadtree class
• loaded: tile is ready to be displayed

Example: see TileProvider.js, TileMesh.js and NodeProcess.js

TODO list:

• Minor changes to BrowseTree/NodeProcess
• Restore command cancellation
• Clean up (remove dead and commented code)
• Merge with current master
• Reuse WMTS for imagery
• Call browse() when a tile has been updated (loading is stuck until the camera moves otherwise) Fixed, I wasn't using the providers correctly (not returning promises)
• Clean globe API to reflect these changes

Why these changes?

• The goal is to better define the place where the node's states are changed. Visibility states (visible and material.visible) are changed only in NodeProcess, loading states (loaded, cullable) are changed in the tile class. Only pending is a bit out of place (in interfaceCommander and tile classes, maybe this could be solved by using a callback after each command is executed) Fixed using callbacks. This kind of separation helps when trying to understand what is going wrong and where when debugging
• Creating tiles early offers more control on the loading of data and cancelling of commands
• Moves some dependencies that seemed out of place in other classes
• The getStatus function is more flexible (easy way to define the order in which the data must be loaded or to ask for data to be reloaded/changed...)

Edit: If you want to test the branch, you will see that there are still problems with the texturing of the globe. There are weird things happening with the UVs of the texture that I didn't quite investigate yet. I temporarily fixed it by modifying the shader (which I didn't commit). I attached the diff if you want to try the branch: shader.txt

[gchoqueux]

• Tiles have an update() : replace by getStatut() or progress(), update() is to apply changes;
• the requests will be formulated in nodeProcess (from up(node) or upSubLayer(node))
• cullable and divided are not necessary. they must be removed;
• loaded is also redundant, with getStatut() or progress();

[Jeremy-Gaillard]

1. Done. Update is now getStatus.
2. I'm not sure I understand. You mean the command (or at least the command's parameters) should be built in a separate function in nodeProcess?
3. Removed divided. Cullable will come next
4. Done.

Open question: should getStatus return an array of statuses instead of just a string (e.g. ["geometry", "imagery"])? This way we could launch parallel commands, be in a ready state (displayable) without having to load everything, etc.

[gchoqueux]

1. QuadTree.up(node) move to nodeProcess.up(node)

getStatut() can return empty or partially empty layers

[Jeremy-Gaillard]

I added a callback after each command execution. By default, it only sets the pending attribute to false, but it can be easily be expanded by passing a function to InterfaceCommander.request().
For example, in NodeProcess.js you could do:

...
var cb = function(type, id) { return function() {console.log(type, id);};};
this.interCommand.request(updateType, node, layer, {callback: cb(updateType, node.id)});

Result:

"geometry" 24
"elevation" 24
"geometry" 35
...

Can be handy for debugging.

I still want to change something before merging with the master. Each time a promise is resolved, I have to test whether a tile has been disposed of or not (in the time a server request is completed, the tile may have been deleted). This means I have to add a test each time I rely on promises, for example, in TileProvider.js:

return this.providerElevationTexture.getElevationTexture(tile.tileCoord, elevationlayerId).then(function(terrain) {
  if(this.disposed) return;
  this.setTextureElevation(terrain);
}.bind(tile));

I'd like this process to be automated, so when implementing a new Provider, the programmer doesn't have to worry about it. Does anyone have an idea on how to do this elegantly (maybe @tbroyer )?

[Jeremy-Gaillard]

I'd like to have some feedback on the PR. Most of the developers will be impacted by this work since it changes the way providers and tiles interact, so it is important for the project as a whole that other developers share their opinion on this matter.

Latest development:

Changed getStatus() to return an array of data types. Added a ready() function to the node to separate data request (from getStatus) and readiness.

I restored WMTS imagery, but I had to disable texture upscaling in the process (using the parent's texture while waiting for the right resolution to load). However, I think this part of the code needs to be changed (and moved out of WMTS_provider) because it seemed overly complicated.

I'm still trying to find a way to remove the cullable attribute and to solve the issue mentionned in my previous comment.

[Jeremy-Gaillard]

ping @gchoqueux @vmora @qdnguyen @nosy-b for feedback on what is basically the new low-level API.

[vmora]

one liner ?

arrayTasks.push(this.providerMap[command.layer.id].executeCommand(command).then(command.callback));

[vmora]

seems reasonable.

[vpicavet]

@gchoqueux @qdnguyen @nosy-b Any update on reviewing this ? @peppsac will have a look at it soon too, to get it finally merged

[Jeremy-Gaillard]

Still a few issues I'm working on. There are a few regression that need fixing, most notably the elevation not loading when zoomed in a lot.

[nosy-b]

Didn't check deeply the modifications yet but It looked pretty slow compare to the master when we tried yesterday with @gchoqueux and as you mentioned, elevation was not loaded closer to the ground.

[peppsac]

I'm going to rebase this PR on master, then I'll work on fixing the 2 identified regressions (elevation and slowness).

[peppsac]

@Jeremy-Gaillard maybe we could close this PR as most of its content/ideas has been dispatched in smaller PR?