Performance best practices for frequent data updates + working with static and realtime datasets

[JamesRexMiller4]

My company is working with deck.gl but we have encountered some bottlenecks that I feel have to be more the fault of my implementation than any limitations of the library.

Brief overview:

we have multiple large datasets, each consisting of 1000s of points
we render only one dataset at a time, and each point is rendered typically as a column
we need to regularly append to the visible dataset
we need to switch between the datasets

We are thinking about refactoring and only sending points that are within view given the map lat/long coordinates. At further zoomed out levels we would do some data smoothing so that given a particular number of pixels available we only send what would occupy a single pixel and achieve the visual representation without excess.

Would you recommend this approach, or is this unnecessary?

Other questions:

Is there an upper limit to how large of a dataset(s) you can work with when updating at a regular interval?

Is it more performant to have lots of layers with small datasets (point per layer), or one layer with a large dataset, or does it make no difference?

In a scenario where the data is static, would it be better to opt for a single layer?

[Pessimistress]

We are thinking about refactoring and only sending points that are within view given the map lat/long coordinates. At further zoomed out levels we would do some data smoothing so that given a particular number of pixels available we only send what would occupy a single pixel and achieve the visual representation without excess.

TileLayer is for this exact purpose.

Run time performance is browser/hardware bound, but low thousands should not have a visible overhead. If you haven't, check out these documentations:

• Performance optimization
• _dataDiff