Skip to content

Commit

Permalink
[PART-3] CPUAggregation Refactor : Remove CPU Aggregation from GPUGri…
Browse files Browse the repository at this point in the history
…dAggregator (#3953)

* GPUAggregator: Remove CPU Aggregation support
* Consolidate CPU Aggregation
* ContourLayer: only perform shallow compare for contours prop, individual triggers not needed anymore
* Conver matrix multiply to translation/scaling
* Remove fp64 flag and dependency on project64 module
* Unify screenspace projections, remove usage of gridTransformMatrix
* Use 3D position for CPU and GPU aggregation
* GPUAggregation: use 64 bit position attribute
  • Loading branch information
1chandu committed Dec 7, 2019
1 parent 27f6cec commit 39b105f
Show file tree
Hide file tree
Showing 34 changed files with 1,302 additions and 1,640 deletions.
2 changes: 1 addition & 1 deletion dev-docs/RFCs/v8.0/grid-aggregation-attributemanager.md
Expand Up @@ -28,7 +28,7 @@ Introduce new composite layer `AggregationLayer`, which owns `GPUGridAggregator`

#### Attributes setup

`AttributeManager` is only available for `Layer` and it's subclasses and not available for a `CompositeLayer`. We can create an instance of `AttributeManger` in `AggregationLayer` and provide it to all aggregation layers which can be used for all offline renderings performed for aggregation.
`AttributeManager` is only available for `Layer` and it's subclasses and not available for a `CompositeLayer`. We can create an instance of `AttributeManager` in `AggregationLayer` and provide it to all aggregation layers which can be used for all offline renderings performed for aggregation.

#### Model setup

Expand Down
3 changes: 1 addition & 2 deletions dev-docs/RFCs/vNext/fixed-frame-coordinates-rfc.md
Expand Up @@ -19,7 +19,7 @@ However, fixed frame geospatial coordinates are also well-defined and have a com

During our collaboration with the Cesium team around 3D tiles in loaders.gl and math.gl, we had to implement support for fixed frame geospatial coordinates in math.gl (for Ellipsoid math and WGS84 transformations from cartesian to cartographic coordinates).

This puts us in a position where we now have both the necessary support libraries, an initial use case for, and a more mature understanding of fixed frame geospatial coodinates, so it makes sense to create an RFC to dig into what adding fixed frame coordinate support directly to deck.gl would mean.
This puts us in a position where we now have both the necessary support libraries, an initial use case for, and a more mature understanding of fixed frame geospatial coordinates, so it makes sense to create an RFC to dig into what adding fixed frame coordinate support directly to deck.gl would mean.


## Overview
Expand Down Expand Up @@ -112,4 +112,3 @@ new Layer({
### Proposal: Built-in Frustum Culling Functionality in deck.gl

Would be easier to implement in a fixed coordinate system.

20 changes: 9 additions & 11 deletions docs/layers/contour-layer.md
Expand Up @@ -14,6 +14,12 @@
import DeckGL from '@deck.gl/react';
import {ContourLayer} from '@deck.gl/aggregation-layers';

const CONTOURS = [
{threshold: 1, color: [255, 0, 0, 255], strokeWidth: 1}, // => Isoline for threshold 1
{threshold: 5, color: [0, 255, 0], strokeWidth: 2}, // => Isoline for threshold 5
{threshold: [6, 10], color: [0, 0, 255, 128]} // => Isoband for threshold range [6, 10)
];

const App = ({data, viewport}) => {

/**
Expand All @@ -26,11 +32,7 @@ const App = ({data, viewport}) => {
const layer = new ContourLayer({
id: 'contourLayer',
// Three contours are rendered.
contours: [
{threshold: 1, color: [255, 0, 0, 255], strokeWidth: 1}, // => Isoline for threshold 1
{threshold: 5, color: [0, 255, 0], strokeWidth: 2}, // => Isoline for threshold 5
{threshold: [6, 10], color: [0, 0, 255, 128]} // => Isoband for threshold range [6, 10)
],
contours: CONTOURS,
cellSize: 200,
getPosition: d => d.COORDINATES,
});
Expand Down Expand Up @@ -107,18 +109,14 @@ Array of objects with following keys

* `zIndex` (Number, optional) : Defines z order of the contour. Contour with higher `zIndex` value is rendered above contours with lower `zIndex` values. When visualizing overlapping contours, `zIndex` along with `zOffset` (defined below) can be used to precisely layout contours. This also avoids z-fighting rendering issues. If not specified a unique value from `0` to `n` (number of contours) is assigned.

NOTE: Like any other layer prop, a shallow comparison is performed on `contours` prop to determine if it is changed. This prop should be set to an array object, that changes only when contours need to be changed.

##### `zOffset` (Number, optional)

* Default: `0.005`

A very small z offset that is added for each vertex of a contour (Isoline or Isoband). This is needed to control the layout of contours, especially when rendering overlapping contours. Imagine a case where an Isoline is specified which is overlapped with an Isoband. To make sure the Isoline is visible we need to render this above the Isoband.

##### `fp64` (Boolean, optional)

* Default: `false`

Whether the aggregation should be performed in high-precision 64-bit mode. Note that since deck.gl v6.1, the default 32-bit projection uses a hybrid mode that matches 64-bit precision with significantly better performance.

### Data Accessors

##### `getPosition` ([Function](/docs/developer-guide/using-layers.md#accessors), optional)
Expand Down
23 changes: 4 additions & 19 deletions docs/layers/gpu-grid-layer.md
Expand Up @@ -8,7 +8,7 @@
# GPUGridLayer (WebGL2)

The `GPUGridLayer` renders a grid heatmap based on an array of points.
It takes the constant cell size, aggregates input points into cells. When possible aggregation is performed on GPU. The color
It takes the constant cell size, aggregates input points into cells. This layer performs aggregation on GPU hence not supported in non WebGL2 browsers. The color
and height of the cell is scaled by number of points it contains.

`GPUGridLayer` is one of the sublayers for [GridLayer](/docs/layers/grid-layer.md) and is only supported when using `WebGL2` enabled browsers. It is provided to customize GPU Aggregation for advanced use cases. For any regular use case, [GridLayer](/docs/layers/grid-layer.md) is recommended.
Expand All @@ -30,7 +30,6 @@ const App = ({data, viewport}) => {
*/
const layer = new GPUGridLayer({
id: 'gpu-grid-layer',
gpuAggregation: true,
data,
pickable: true,
extruded: true,
Expand All @@ -51,7 +50,9 @@ const App = ({data, viewport}) => {

**Note:** The `GPUGridLayer` at the moment only works with `COORDINATE_SYSTEM.LNGLAT`.

**Note:** This layer is similar to [CPUGridLayer](/docs/layers/cpu-grid-layer.md) but supports aggregation on GPU when possible. Check below for more detailed differences of this layer compared to `CPUGridLayer`.
**Note:** GPU Aggregation is faster only when using large data sets (point count is more than 500K), for smaller data sets GPU Aggregation could be potentially slower than CPU Aggregation.

**Note:** This layer is similar to [CPUGridLayer](/docs/layers/cpu-grid-layer.md) but performs aggregation on GPU. Check below for more detailed differences of this layer compared to `CPUGridLayer`.


## Installation
Expand Down Expand Up @@ -137,22 +138,6 @@ Cell elevation multiplier. The elevation of cell is calculated by

Whether to enable cell elevation. Cell elevation scale by count of points in each cell. If set to false, all cell will be flat.

##### `fp64` (Boolean, optional)

* Default: `false`

Whether the aggregation should be performed in high-precision 64-bit mode. Note that since deck.gl v6.1, the default 32-bit projection uses a hybrid mode that matches 64-bit precision with significantly better performance.

##### `gpuAggregation` (bool, optional)

* Default: true

When set to true and browser supports GPU aggregation, aggregation is performed on GPU. GPU aggregation can be 2 to 3 times faster depending upon number of points and number of cells.

**Note:** GPU Aggregation requires WebGL2. When `gpuAggregation` is set to true and browser doesn't support WebGL2, aggregation falls back to CPU.

**Note:** GPU Aggregation is faster only when using large data sets (point count is more than 500K), for smaller data sets GPU Aggregation could be potentially slower than CPU Aggregation.

##### `material` (Object, optional)

* Default: `true`
Expand Down
12 changes: 7 additions & 5 deletions modules/aggregation-layers/src/aggregation-layer.js
Expand Up @@ -48,7 +48,7 @@ export default class AggregationLayer extends CompositeLayer {
if (shaders && shaders.defines) {
shaders.defines.NON_INSTANCED_MODEL = 1;
}
this._updateShaders(shaders);
this.updateShaders(shaders);
}

// Explictly call to update attributes as 'CompositeLayer' doesn't call this
Expand Down Expand Up @@ -83,7 +83,11 @@ export default class AggregationLayer extends CompositeLayer {
return moduleSettings;
}

_isAggregationDirty(opts) {
updateShaders(shaders) {
// Default implemention is empty, subclasses can update their Model objects if needed
}

isAggregationDirty(opts) {
if (this.state.dataChanged || opts.changeFlags.extensionsChanged) {
return true;
}
Expand All @@ -99,9 +103,7 @@ export default class AggregationLayer extends CompositeLayer {
);
}

_updateShaders(shaders) {
// Default implemention is empty, subclasses can update their Model objects if needed
}
// Private

// override Composite layer private method to create AttributeManager instance
_getAttributeManager() {
Expand Down
35 changes: 35 additions & 0 deletions modules/aggregation-layers/src/aggregation-layer.md
@@ -0,0 +1,35 @@
# AggregationLayer

All of the layers in `@deck.gl/aggregation-layers` module perform some sort of data aggregation. All these layers perform aggregation with different parameters (CPU vs GPU, aggregation to rectangular bins vs hexagon bins, world space vs screen space, aggregation of single weight vs multiple weights etc).

`AggregationLayer` and `GridAggregationLayer` perform most of the common tasks for aggregation with flexibility of customizations. This document describes what `AggregationLayer` does and how to use it in other aggregation layers.


`AggregationLayer` is subclassed form `CompositeLayer` and all layers in `@deck.gl/aggregation-layers` are subclassed from this Layer.

### Integration with `AttributeManager`

This layer creates `AttributeManager` and makes it available for its subclasses. Any aggregation layer can add attributes to the `AttributeManager` and retrieve them using `getAttributes` method. This enables using `AttributeManager`'s features and optimization for using attributes. Also manual iteration of `data` prop can be removed and attributes can be directly set on GPU aggregation models or accessed directly for CPU aggregation.

Example: Adding attributes to an aggregation layer

```
const attributeManager = this.getAttributeManager();
attributeManager.add({
positions: {size: 3, accessor: 'getPosition'},
color: {size: 3, accessor: 'getColorWeight'},
elevation: {size: 3, accessor: 'getElevationWeight'}
});
```

### updateState()

During update state, Subclasses of `AggregationLayer` must first call 'super.updateState()', which calls

- `updateShaders(shaders)` : Subclasses can override this if they need to update shaders, for example, when performing GPU aggregation, aggregation shaders must be merged with argument of this function to correctly apply `extensions`.

- `_updateAttributes`: This checks and updates attributes based on updated props.

### Checking if aggregation is dirty

Constructor, takes an array of props, `aggregationProps`, and a private method `isAggregationDirty()` is provided that returns `true` when any of the props in `aggregationProps` are changed. Subclasses can customize this to desired props by providing `aggregatinProps` array.

0 comments on commit 39b105f

Please sign in to comment.