scattergl triggers WebGL errors when using arrays of symbol, color, or size

[brian428]

I'm experimenting with the Plotly.js scattergl chart, and seem to be running into issues when trying to use arrays of symbols, colors, and sizes to allow per-point styling.

Take a look at this fiddle.

When using only a single symbol, size or color, the chart renders fine. But if you comment out the single size and color (lines 48-49), and comment in the use of arrays of values, WebGL dies and the following is shown in the console:

GL_INVALID_OPERATION : glGenSyncTokenCHROMIUM: fence sync must be flushed before generating sync token

[.CommandBufferContext]GL ERROR :GL_OUT_OF_MEMORY : BackFramebuffer::Destroy: <- error from previous GL command

WebGL: CONTEXT_LOST_WEBGL: loseContext: context lost

The same thing happens in Firefox as well.

Does scattergl not allow for arrays of values for symbol, style and color? Is there some other way of setting styles on each point that I'm not aware of?

[etpinard]

Hmm. Strange. I was able it to work in https://jsfiddle.net/chs83a7t/2/

Could you update your fiddle to a case that actually outputs the errors you mentioned above?

Thank you very much.

[brian428]

Updated test.

So it looks like it may be machine-specific in some way. I tried it on my desktop machine and it worked, but from my laptop it fails. I honestly don't know enough about WebGL to know why it would work on one machine and fail on another.

Your original fiddle works on both the desktop and the laptop. The problem only seems to manifest once the number of points goes up and arrays of values are used for symbol/style/color instead of single values. Obviously the desktop has a much more powerful GPU than the laptop, but it seems like drawing a graph like this wouldn't kill the GPU. But it sure seems like there is something very different going on under the hood when arrays of values are used instead of single values.

For reference, here's a screenshot of what I'm seeing:

If you can think of anything, or need any more info, just let me know. Thanks.

[brian428]

Also, in case it helps, here is the Chrome GPU log showing the problems.
gpu.zip

[brian428]

I did notice the GPU driver is quite old. This is a corporate-managed machine so I'm not sure if I'll be able to update it myself, but I may also give that a shot.

[etpinard]

I honestly don't know enough about WebGL to know why it would work on one machine and fail on another.

Our WebGL trace types do use some cutting edge specs. This is not really surprising. Thank you very much for this report.

[brian428]

OK, I'll update once I'm (hopefully) able to try it using more current GPU drivers.

[brian428]

Just to follow up here, I was able to install the latest Nvidia drivers, but the WebGL error still appears when using arrays of per-point values for color, symbol or size. Unfortunately, this is a showstopper for me, since I obviously can't move forward with using Plotly.js as a replacement for Highcharts to due this problem.

I'm not sure what else I can try to do here, but if you can think of any information I can offer please let me know.

In case it helps, I'll attach a new Chrome GPU log.
gpu-new.zip

[brian428]

Also, perhaps I should ask: if you need to be able to specify things like color, size and symbol for each point, is there some other way to do this that I'm not aware of? I'm still not entirely clear on why using a single color/size/symbol for all of the points works fine, but using arrays for these causes things to blow up.

[etpinard]

if you need to be able to specify things like color, size and symbol for each point, is there some other way to do this that I'm not aware of?

You could use multiple traces, but that will lead to a significant slow down. I wouldn't recommend it.

I'm still not entirely clear on why using a single color/size/symbol for all of the points works fine, but using arrays for these causes things to blow up.

It should work. This is definitely a bug on our part. Unfortunately, it is machine/browser dependent and won't be an easy fix.

[brian428]

Just thought I would add that part of the issue might be tied to memory use. Here are two screenshots of the Chrome task manager. The first one is my scattergl test without per-point values for color/size/symbol, and the second is when specifying per-point arrays for color/size/symbol. As you can see, memory use really explodes in the second case.

[etpinard]

cc @dfcreative

[dy]

That is the issue of gl-scatter2d-fancy, actually. We can reproduce it in this example by changing POINT_COUNT to 1e5.

That happens because plotly sees array of colors for markers and switches renderer from gl-scatter2d, which is performant one, to gl-scatter2d-fancy, which can style individual markers better.

Fancy scatter creates additional colorBuffer, storing values per marker vertex. Each value takes additional 32bits per item, or 1Gb+ of graphic card memory for 1e5 points.

As a possible solution would be using compressed textures for colors, instead of buffer. Or trying to filter repeating colors. Or just reconsider the need in that many colors.

Also consider FloatArrays length limit, which varies, but <= 2³², or practically < 1e8.

Though it will not remove performance lags of plotly, even comparing with gl-scatter2d standalone.

[brian428]

To be clear, I don't need different colors for every single point. But I do need the ability to change the colors, sizes, and/or shapes for subsets of points (e.g. to mark/highlight points, etc.). This seems like a pretty basic requirement to me. Unless there's some way I'm not aware of to do this, arrays of colors/sizes/shapes is the only possible option, correct?

[etpinard]

Unless there's some way I'm not aware of to do this, arrays of colors/sizes/shapes is the only possible option, correct?

Do you know before hand which points are going to be highlighted? In this case, you can setup two scattergl traces, one for each color.

[brian428]

Unfortunately, no. I've got an arbitrary number of series (generally 2 to 8 or so), with an arbitrary number of points (generally 5,000-50,000) per series. Each series has an initial color/size/shape, but the user can box-select any points (which requires highlighting them in some way), and then tag/group them with a label, and a custom color/size/shape. The user can then filter based on series, tags, etc.

So while there won't commonly be more than 10 or 15 different color/size/shape values, the actual number is variable, and which points they're applied to is variable (obviously).

WebGL would seem ideal for this. Looking at large 3D graph/bubble/scatter plots made with something like three.js doesn't produce any excessive memory usage. So I assume this should all be possible, but it sounds like the way plotly.js is handling this is the underlying issue, correct?

I'm not sure what this means, or if it indicates that a fundamental re-work is needed (either in the way color/size/shape is stored/cached, or for the WebGL pieces in general)? Plotly's WebGL features seem targeted at large data sets, but 100,000 points is not an outlandish data set size. Yes, it's big, but it's not millions or tens of millions. If variable color/size/shape for larger data sets is unworkable, this probably needs to be highlighted somewhere, since it's a pretty critical limitation.

Thoughts?

[brian428]

As a random thought, what about allowing for some sort of point renderer function? So that instead of forcing Plotly.js to track every single color/size/shape as separate values, it can optionally invoke a function to determine these values on a per-point basis? That way, the work for determining these values can be offloaded to the developer, to optimize however they see fit?

[brian428]

Just curious if there was any ETA on when this might be addressed (or if it's probably not coming any time soon)? It seems like allowing a hook for a render function might be a decent short-term option? Basically, it would leave it up to the developer to determine how to create the formatting configuration for each point (and how to optimize that decision process).

I'm not sure how the WebGL piece of Plotly.js is currently implemented, but I noticed that Three.js seems to handle this sort of thing pretty well. They have a "particle" demo that shows a lot of "points" of varying sizes and colors (https://threejs.org/examples/#webgl_points_random). And seems to keep memory use quite low:

The downside is that Three.js seems to be a very low-level option, so I imagine that trying to wrap it within Plotly.js wouldn't be a simple task. No idea if that's worth exploring, but just thought I'd mention it.

[etpinard]

@brian428

Just curious if there was any ETA on when this might be addressed

It's something that we'll fix during #949.

That said, I can't guarantee any serious work on this issue before then. Unless @dfcreative would be interested in looking into this.

[dgiovann89]

@brian428 i have the same your problems(I need to represent min 20k max 500k points with about 10 or 20 colors or shapes) ...finally you have solved with plotly.js, three.js or with other methods / libraries?

[brian428]

@dgiovann89: unfortunately, no, not yet.

[dy]

@etpinard I can try to refactor scattergl to group render passes by color, that may help with specifically that case

[etpinard]

that would be great @dfcreative 👍