setData() and smart diffing

[peterqliu]

Currently, the only way to bind and update data to layers is via setData. Regardless of how new data differs from the old, this involves a full rerender. The downsides are familiar:

1. This is slow. Noticeable delay before update happens.

2. Makes continuity difficult. If we want to show a vehicle marker moving down the street, it involves creating and destroying markers in a repeated fashion, rather than animating one marker to new positions (to be fair, if we made # 1 extremely fast, the two become indistinguishable).

The ideal solution would be able to

1. Handle atomic changes to source data in a way that's smart and fast, and
2. Distinguish new "objects" in the data (e.g., new cars on the road), from merely updated states of existing objects (cars that have moved)

The obvious predecessor here is D3's enter/update/exit pattern.

cc/ @kelvinabrokwa @tmcw @jfirebaugh

[tmcw]

The d3 model is heavily influenced by the SVG/HTML rendering model: it
really should not be the first thought for a system in GL, which has a
fundamentally different approach to data and rendering.

On Saturday, July 25, 2015, Peter Liu notifications@github.com wrote:

Currently, the only way to bind and update data to layers is via setData
https://www.mapbox.com/mapbox-gl-js/api/#geojsonsource/setdata.
Regardless of how new data differs from the old, this involves a full
rerender of the map. The downsides are familiar:

1. This is slow. Noticeable delay before update happens.
2. Makes continuity difficult. If we want to show a vehicle marker
   moving down the street, it involves creating and destroying markers in a
   repeated fashion, rather than animating one marker to new positions (though
   if we make 1) fast enough, this becomes moot).

The ideal solution would be able to

1. Handle atomic changes to source data in a way that's smart and fast, and
2. Distinguish new "objects" in the data (e.g., new cars on the road),
   from merely updated states of existing objects (cars that have moved)?

The obvious predecessor here is D3's enter/update/exit
http://bost.ocks.org/mike/join/ pattern.

cc/ @kelvinabrokwa https://github.com/kelvinabrokwa @tmcw
https://github.com/tmcw @jfirebaugh https://github.com/jfirebaugh

—
Reply to this email directly or view it on GitHub
#1391.

[lucaswoj]

If we built a system that would allow us to make incremental changes to the GL buffers (which is possible), we would still have to do an O(n) copy of the entire buffer to the GPU on every change. Let's focus on making GL fast enough to create fresh buffers at a framerate suitable for animation and keep this idea on the back burner.

[mourner]

Also see upstream: mapbox/geojson-vt#26

[lucaswoj]

I don't see this feature fitting into GL JS's API. We should continue trying to make setData faster.