New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Indexed vector sources #1377

Closed
wants to merge 28 commits into
base: master
from

Conversation

Projects
None yet
8 participants
@tmcw
Contributor

tmcw commented Jul 19, 2015

Style, context, polish

  • @chelm is there any git history for the initial development on this branch?
  • demo/mapbox-gl-dev.js and demo/streets-mobile.json probably shouldn't be checked in
  • esri- files follow a dashed naming convention that diverges from the underscored naming convention of the rest of the lib
  • esri- code needs JSDoc strings like the rest of the lib
  • esri-tile-pyramid differs more in functionality than proprietariness to the TilePyramid code, and it is exposed via GL styles as indexedVector: should it be titled IndexedTilePyramid instead?
  • The new tile pyramid contains a lot of code verbatim from the existing one: parts in common should be unified by inheritance or modularization.

indexedVector tech

  • Are there any benchmarks or tests that have informed your decision to have an indexed tile pyramid? I can imagine there being a performance benefit, but would prefer concrete numbers to point to. Also, is the avoidance of 404s currently commented-out?

merge blockers

  • Tests
  • vector-tile-js dependency needs open-source release Esri#1 (comment)
  • mapbox-gl-style-spec dependency needs open source release

High-level there are three outcomes:

  1. indexedVector is a performance optimization. If it can't be confirmed as an improvement, then we may not accept it.
  2. If indexedVector's performance wins are clear-cut then it's possible we'll accept it as a default and integrate it into v8 rather than having two ways to load vector tiles
  3. For routes in the middle, we'll help get this branch landed

/cc @chelm @jcardonadcdev @ycabon

@mourner mourner added the in progress label Jul 19, 2015

while (ids.length) {
id = ids.pop();
tile = TileCoord.fromID(cursorId);
index = tile.children(this.maxzoom).map(pluckId).indexOf(id);

This comment has been minimized.

@tmcw

tmcw Jul 19, 2015

Contributor

Is this hot code? Seems like the creation of function, a parallel array, and then another o(n) indexOf search would be a potential perf problem.

@tmcw

tmcw Jul 19, 2015

Contributor

Is this hot code? Seems like the creation of function, a parallel array, and then another o(n) indexOf search would be a potential perf problem.

This comment has been minimized.

@chelm

chelm Jul 20, 2015

Inefficient code for sure, can be much cleaner. A single loop would do what we need.

@chelm

chelm Jul 20, 2015

Inefficient code for sure, can be much cleaner. A single loop would do what we need.

};
// request the tile parentID if it exists
if (tile.parentId) {

This comment has been minimized.

@tmcw

tmcw Jul 19, 2015

Contributor

afaict this is the first change I've seen that modifies existing behavior rather than extending it with a new source type.

@tmcw

tmcw Jul 19, 2015

Contributor

afaict this is the first change I've seen that modifies existing behavior rather than extending it with a new source type.

This comment has been minimized.

@chelm

chelm Jul 20, 2015

Right. If a tile has a parentId the url for the tile request needs to be used in place of the url built from tile.id

@chelm

chelm Jul 20, 2015

Right. If a tile has a parentId the url for the tile request needs to be used in place of the url built from tile.id

@chelm

This comment has been minimized.

Show comment
Hide comment
@chelm

chelm Jul 20, 2015

@tmcw Thanks for starting this. The intention of checking the original code in (all the esri-* files) was to have a temp reference to how the original code was changed. I'll remove it since what I've done here is actually remove the concept of an "indexedVector" and have changed the behavior of the existing "vector" data sources.

I'll explain the changes better with line notes, and will sync the PR to remove unneeded cruft.

chelm commented Jul 20, 2015

@tmcw Thanks for starting this. The intention of checking the original code in (all the esri-* files) was to have a temp reference to how the original code was changed. I'll remove it since what I've done here is actually remove the concept of an "indexedVector" and have changed the behavior of the existing "vector" data sources.

I'll explain the changes better with line notes, and will sync the PR to remove unneeded cruft.

Show outdated Hide outdated js/source/source.js
@@ -25,6 +25,8 @@ function TilePyramid(options) {
this.maxzoom = options.maxzoom;
this.roundZoom = options.roundZoom;
this.reparseOverscaled = options.reparseOverscaled;
// esri/chelm
this.index = options.index;

This comment has been minimized.

@chelm

chelm Jul 20, 2015

make sure the index is accessible on the tile pyramid

@chelm

chelm Jul 20, 2015

make sure the index is accessible on the tile pyramid

@@ -271,6 +273,10 @@ TilePyramid.prototype = {
var zoom = coord.z;
var overscaling = zoom > this.maxzoom ? Math.pow(2, zoom - this.maxzoom) : 1;
tile = new Tile(wrapped, this.tileSize * overscaling);
// esri/chelm
if (this.index) {
tile.parentId = this.indexSearch(coord.id);

This comment has been minimized.

@chelm

chelm Jul 20, 2015

When a tile is added to the map, we search the pyramid's index for a "parentId". A parentId should only exist when a requested tile is not found in the tile index. The index is searched recursively until the next lowest zoom level tile is found.

After explaining that I think the name "parentId" should be changed to be a bit more representative of what it is...

@chelm

chelm Jul 20, 2015

When a tile is added to the map, we search the pyramid's index for a "parentId". A parentId should only exist when a requested tile is not found in the tile index. The index is searched recursively until the next lowest zoom level tile is found.

After explaining that I think the name "parentId" should be changed to be a bit more representative of what it is...

Show outdated Hide outdated js/source/worker.js
Show outdated Hide outdated js/source/worker_tile.js
@chelm

This comment has been minimized.

Show comment
Hide comment
@chelm

chelm Jul 28, 2015

Just to comment on the status here: The team inside Esri is focusing on cooking up some benchmarks on using a tile index across our stack. This will help us understand the impact of the index on tile generation (less tiles) and the impact at render time (data clipping vs requesting/rendering very small tiles). It may be some time still, but everyone is eager to keep this moving forward.

chelm commented Jul 28, 2015

Just to comment on the status here: The team inside Esri is focusing on cooking up some benchmarks on using a tile index across our stack. This will help us understand the impact of the index on tile generation (less tiles) and the impact at render time (data clipping vs requesting/rendering very small tiles). It may be some time still, but everyone is eager to keep this moving forward.

@odoe

This comment has been minimized.

Show comment
Hide comment
@odoe

odoe Oct 13, 2015

Here is a gist that discusses the concept of the Indexing of Vector Tiles and the justification for Clipping.

https://gist.github.com/odoe/ce6a150658526901ef27#file-vector-tile-pr-md

cc @tmcw @ycabon @jcardonadcdev

odoe commented Oct 13, 2015

Here is a gist that discusses the concept of the Indexing of Vector Tiles and the justification for Clipping.

https://gist.github.com/odoe/ce6a150658526901ef27#file-vector-tile-pr-md

cc @tmcw @ycabon @jcardonadcdev

@ycabon

This comment has been minimized.

Show comment
Hide comment
@ycabon

ycabon commented Oct 13, 2015

@jcardonadcdev

This comment has been minimized.

Show comment
Hide comment
@jcardonadcdev

jcardonadcdev Oct 13, 2015

Tests used to compare processing times for indexed tile vs. non-indexed tile - https://github.com/jcardonadcdev/vt-performance-tests

jcardonadcdev commented Oct 13, 2015

Tests used to compare processing times for indexed tile vs. non-indexed tile - https://github.com/jcardonadcdev/vt-performance-tests

@tmcw

This comment has been minimized.

Show comment
Hide comment
@tmcw

tmcw Aug 15, 2016

Contributor

The Esri side of this fork hasn't been updated since November 2015, and Mapbox GL JS has changed massively since then. It doesn't look like there are any active branches on the Esri project anymore.

Happy to take a look at an updated branch that resolves some of the initial issues outlined in this PR! Closing this specific PR as stale.

Contributor

tmcw commented Aug 15, 2016

The Esri side of this fork hasn't been updated since November 2015, and Mapbox GL JS has changed massively since then. It doesn't look like there are any active branches on the Esri project anymore.

Happy to take a look at an updated branch that resolves some of the initial issues outlined in this PR! Closing this specific PR as stale.

@tmcw tmcw closed this Aug 15, 2016

@chelm

This comment has been minimized.

Show comment
Hide comment
@chelm

chelm commented Aug 15, 2016

😞

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment