Data Management RFC #1158
Data Management RFC #1158
Conversation
| ## Implementation | ||
|
|
||
| A `DataManager` class that supports the following methods: | ||
| - `addData(id, data, accessors)`: add a data set to the manager, optionally with accessors to create particular views. |
There was a problem hiding this comment.
I feel that this is just moving the complexity from AttributeManager to DataManager. I'd prefer a minimal class that is just ResourceManager and not deal with data creation.
There was a problem hiding this comment.
The issue is that the caching requires identifying when the same data is being used. If the accessors are too much, I could just make it take a typed array or an image (i.e. all the parsing would be done in deck, and then the value arrays would be registered with the DataManager).
There was a problem hiding this comment.
In deck.gl we do update a typed array in-place before updating the buffer. It's a poor indicator for using cached result since the content of the same typed array and the same buffer may be out of sync.
There was a problem hiding this comment.
Hmmm... maybe it would be better to make the API closer to the ProgramManager then. Something like dm.addBuffer('myBuffer', buffer), where the application would be responsible for providing a key that uniquely identifies a buffer. And all the other operations would be based on buffer ids. Might be tricky to do caching for data, but things like Geometry attributes would be easy to cache.
| dm.addData('anotherImage', image3); // Accessors not provided because the only view is the entire data set | ||
|
|
||
| const positionBuffer = dm.getBuffer('myTable', 'positions'); | ||
| const positionBuffer2 = dm.getBuffer('myTable', 'positions'); // Gets cached version of same buffer |
There was a problem hiding this comment.
In deck.gl Attribute class we need the following operations:
- Allocate an empty buffer of a specific size
- Create a new buffer from TypedArray
- Resize an existing buffer (keep the data but allocate bigger size)
- Partially update an existing buffer
- Copy a chunk from one buffer to another
There was a problem hiding this comment.
These should all be doable.
|
@Pessimistress Got rid of accessors and added the methods you asked for. |
A system to manage data sets and create buffers or textures from them. Supports caching and re-using resources. Would ideally take over some of the complexity that currently lives in deck.gl's
AttributeManager.