Introducing splom traces #2505

etpinard · 2018-03-28T02:48:29Z

resolves #2372

This is a feature branch built off PRs #2474, #2487, #2492 and #2499 where all splom development should start off from. Other splom-related branches on origin like splom and splom-dev are now obsolete.

cc @dfcreative @alexcjohnson

- only markers for now, reuse scattergl convert logic for markers - use a splom basePlotModule to handle multiple traces and splom-wide field (like regl-line2d grids coming up)

which: - adds only a limited number svg layer in each <g .subplot> - use regl-line2d to draw grids

etpinard · 2018-03-28T02:59:45Z

As of the commit fda1b34,

things that work

splom markers thanks to https://github.com/dfcreative/regl-scattermatrix and their styling thanks to 1c6144e (I haven't tested arrayOk attributes though)
zoom and drag
graph with multiple splom traces
splom traces that generate layout.grid
splom traces that default their corresponding axis title
is pretty fast (more on that soon)

things that do not work

✅ image generation in nw.js (that's why the new splom mock don't have corresponding baselines). Note that image generation via the modebar button appear to work fine. (etienne)
✅ something is off with splom + other cartesian trace types (etienne)
✅ hover (dima)
✅ selection (dima)
✅ legend items styling
~~mode: 'lines'~~ defer to next iteration
✅ splom attributes ~~xdirection, ydirection,~~ showdiagonal, showupperhalf showlowerhalf (or shouldn't that be a flaglist by the way 🤔 )
regl-line2d grid line updates (fixed, but could be cleaned up)
✅ test out marker options
✅ non-linear axes!
tests !
probably more things I can't think about right now

✅ means works now I think.

Here's what the splom_iris mock looks like:

alexcjohnson · 2018-03-28T16:41:26Z

src/plot_api/plot_schema.js

@@ -486,7 +486,7 @@ function getLayoutAttributes() {

        if(!_module.layoutAttributes) continue;

-        if(_module.name === 'cartesian') {
+        if(_module.name === 'cartesian' || _module.name === 'splom') {


Can we do this just based on _module.attr?
if(Array.isArray(_module.attr)) { _module.attr.forEach(... handleBasePlotModule) }

done in 9162265

alexcjohnson · 2018-03-28T17:04:59Z

src/traces/splom/attributes.js

+        role: 'info',
+        editType: 'plot',
+        description: ''
+    },


layout.grid doesn't have an analog of xdirection, do we need that? And in place of ydirection we have roworder: ('top to bottom'|'bottom to top') which feels clearer to me than ('top'|'bottom'). Actually I wonder if these belong in splom attributes at all, or if we should just let layout.grid handle it. It's nice for the simple single-trace case, but once you get two splom traces, seems like only one (the first?) matters, so having one in each trace would be confusing.

Actually I wonder if these belong in splom attributes at all, or if we should just let layout.grid handle it. It's nice for the simple single-trace case, but once you get two splom traces, seems like only one (the first?) matters, so having one in each trace would be confusing.

I agree 100% here. I'm thinking of 🔪 ing xdirection and ydirection. Users won't have a way to plot their splom dimensions from right to left until layout.grid.columnorder is implemented, but I don't think that's a big deal. Any objections?

🔪 in a077c3f

alexcjohnson · 2018-03-28T17:14:25Z

src/traces/splom/attributes.js

+        dflt: true,
+        editType: 'plot',
+        description: ''
+    },


Eventually we will want the diagonals to support various types: scattergl, histogram, box, violin. So I'm wondering if instead of showdiagonal we should have diagonal: {visible: (true|false)} to support future expansion:

diagonal: { visible: (true|false), type: (scattergl|histogram|box|violin), // then attributes specific to each type, as well as some overrides per dimension }

Eventually we will want the diagonals to support various types: scattergl, histogram, box, violin.

Interesting. So you're thinking a single splom trace could generate scatter panels AND histogram or box or violin traces on the diagonal? That would put us on-par with some of the seaborn. Yeah nice. I can't really think of any drawbacks from that approach except this might cause headaches to the workspace team.

a single splom trace could generate scatter panels AND histogram or box or violin traces on the diagonal?

Yes exactly. An extra complication is that this would create even more x or y axes (for the histogram count or box/violin category), depending on the type and orientation of these diagonal plots; but once we do that, it seems like the implementation should be fairly straightforward. But anyway none of that has to happen now, as long as we set up the attributes we DO need now so they'll support this.

hi @alexcjohnson, i'm really interested to use the diagonal type. Is it available to use?

Currently the diagonal can only have scatter type. Follow #2555 for more info, but we do not have any immediate plans for when other types will be added.

oh okay. Thanks

alexcjohnson · 2018-03-28T17:22:29Z

src/traces/splom/attributes.js

+        dflt: true,
+        editType: 'plot',
+        description: ''
+    },


If we get rid of showdiagonal I'm not sure it makes sense to turn these two into a little flaglist - perhaps just leave them as two booleans.

alexcjohnson · 2018-03-28T17:27:50Z

src/traces/splom/defaults.js

+var handleMarkerDefaults = require('../scatter/marker_defaults');
+var handleLineDefaults = require('../scatter/line_defaults');
+var PTS_LINESONLY = require('../scatter/constants').PTS_LINESONLY;
+var OPEN_RE = /-open/;


Again, going to break for symbol numbers

-> move to #2507 thanks!

alexcjohnson · 2018-03-28T17:36:00Z

src/traces/splom/defaults.js

+    var xaxes = coerce('xaxes', xaxesDflt);
+    var yaxes = coerce('yaxes', yaxesDflt);
+
+    // TODO what to do when xaxes.length or yaxes.length !== dimLength ???


I guess if dimLength is longer it's easy to just drop the later dims. What if xaxes.length !== yaxes.length or dimLength is shorter than xaxes or yaxes? The easy thing to do is just truncate them all to the same length, and perhaps until we have a clear use case otherwise we should just do that.

Related though: if you have several splom traces but they don't share all the same dimensions, you could of course handle this by specifying xaxes and yaxes for each, but could you also just pass in an empty dimension for ones you want to skip? If that takes more than 5 minutes to implement though, skip it for now and leave it as an item for future expansion.

Oh you already have dimensions[i].visible: false, nice! That should take care of my "related" point, it'll still reserve x/y axes for these dimensions, right?

The easy thing to do is just truncate them all to the same length, and perhaps until we have a clear use case otherwise we should just do that.

That sounds right to me. I'll go with that unless anyone disagrees.

That should take care of my "related" point, it'll still reserve x/y axes for these dimensions, right?

Yes that should take care of it, but I haven't tried that yet. Putting that on my TODO list 📝

.... thanks for all these comments by the way. I hope you don't find the _hasOnlyLargeSplom flag too hacky.

@alexcjohnson just to confirm what you meant in your related point:

Plotly.newPlot(gd, [{ type: 'splom', dimensions: [{ values: [1, 2, 3] }, { values: [4, 5, 6] }] }, { type: 'splom', dimensions: [{ visible: false }, { visible: false, }, { values: [1, 2, 3] }, { values: [4, 5, 6] }] }])

gives:

done in 07d59d5

... the destroy method for regl-scattermatrix instance is broken, see gl-vis/regl-splom#3

etpinard · 2018-03-28T17:42:07Z

test/image/mocks/splom_iris.json

@@ -691,7 +691,6 @@
  "layout": {
 	"title": "Iris dataset splom",
  	"width": 600,
-	"height": 500,
-	"showlegend": false
+	"height": 500


first cut baselines 🎉

cc @dfcreative

... notice how the legend items aren't showing the correct symbol. Fixing that.

Ignore last comment. Something else fixed that.

etpinard · 2018-04-12T15:57:18Z

Here's what happens with when allowing cross-subplot selections (like we have currently on master):

Three things to notice:

the event data pointNumber values correspond to the pt indices in every dimensions[i].values. I think that's correct. Attribute selectedpoints follows the same rule.
the selection outlines do not match the selected points when selecting across subplots.
multi-select works per subplot, trying to multi-select across subplot doesn't work at the moment.

With commits 71a0395 and 9d0bcfd we have a slightly more consistent (albeit incomplete) behavior:

In brief, with these commits:

we clear selection outlines when selecting in a different subplot than the previous selection
we disallow multi-select across subplots

alexcjohnson · 2018-04-12T22:33:31Z

test/jasmine/tests/plots_test.js

+                'geo',
+                'pie',
+                'ternary'
+            ]);


great test!

etpinard · 2018-04-12T22:35:09Z

with commit a79f5dc splom interactions are now 🔒ed , I'll tag this PR as reviewable.

So, here's an updated pre-merge checklist:

maybe tweak some test values for CI
approval from @alexcjohnson
test @dy regl-line2d update
adapt _splomGrid code for new regl-line2d (if necessary)
release / publish new versions of regl-scattermatrix, regl-scatter2d and regl-line2d
update package.json and package-lock.json
write down first-render time table (10 dims / 20k values, 20/5k, 30/2k, 40/1k, 50/800)

Note that I just made two comments:

arguing for leaving some previous unaddressed in this PR.

alexcjohnson · 2018-04-12T22:52:18Z

As long as all the TODOs added here are linked to the upcoming splom open items issue(s) I agree, they're all fine to omit from this PR. Looking amazing guys! 💃

etpinard · 2018-04-16T16:26:49Z

Here are some first-render benchmarks with showupperhalf: false and diagonal.visible: false:

# of dimensions	# of values per dimension	regl-line2d grid lines?	time [ms]
10	20k	no	400-500
20	5k	yes	600-700
30	2k	yes	900-1000
40	1k	yes	1300-1400
50	800	yes	2000-2300

computed using:

var Nvars = /* fill this in */
var Nrows = /* fill this in */
var dims = []

for(var i = 0; i < Nvars; i++) {
  dims.push({values: []})
  
  for(var j = 0; j < Nrows; j++) {
     dims[i].values.push(Math.random())
  }
}

Plotly.purge(gd);

console.time('splom')
Plotly.newPlot(gd, [{
  type: 'splom',
  dimensions: dims,
  showupperhalf: false,
  diagonal: {visible: false},
}], {width: 2000, height: 2000})
console.timeEnd('splom')

etpinard · 2018-04-16T20:22:41Z

Merging this thing!

etpinard added 13 commits March 27, 2018 22:42

🌴 pie base plot module

44dc3d8

[tmp] pull splom dep from github repo

c56e35e

lib splom

29ca0ed

first cut splom attributes and defaults

8b5af93

adapt grid and cartesian axis logic for sploms

3be9569

first cut splom auto axis type detection

76c64e5

first cut splom plot

59588cc

- only markers for now, reuse scattergl convert logic for markers - use a splom basePlotModule to handle multiple traces and splom-wide field (like regl-line2d grids coming up)

some linting

66494c7

hook in splom drag

20f01cc

add _hasOnlyLargeSploms flag

c5cda6b

which: - adds only a limited number svg layer in each <g .subplot> - use regl-line2d to draw grids

add a few DRY up TODOs

89fb6bb

first cut splom defaults test

9cac919

add two splom mocks (image generation is broken currently)

fda1b34

etpinard added type: feature status: has TODOs labels Mar 28, 2018

etpinard added this to the v1.36.0 milestone Mar 28, 2018

etpinard added 2 commits March 28, 2018 11:25

fixup c5cda6b

cf485b7

fixup PlotSchema.get() for splom

fc6c895

alexcjohnson reviewed Mar 28, 2018

View reviewed changes

Enable hover for splom

7be161b

alexcjohnson reviewed Mar 28, 2018

View reviewed changes

etpinard added 3 commits March 28, 2018 13:41

fixup _scene ref in calcdata stash

80dd61e

first pass splom basePlotModule clean step

0fb65dd

... the destroy method for regl-scattermatrix instance is broken, see gl-vis/regl-splom#3

add splom mocks

d50746b

etpinard commented Mar 28, 2018

View reviewed changes

fixup cross-subplot select logic brought up in 71a0395

9d0bcfd

etpinard added 5 commits April 12, 2018 18:23

fixup relayout for 'grid.(x|y)side'

6fd6919

fix and 🔒 basePlotModules sorting

31e34fa

add splom hover/selection/drag tests

a79f5dc

✏️ splom meta description + register grid in splom index

31836f2

🐄 in various new splom code & comment

6cc634a

alexcjohnson reviewed Apr 12, 2018

View reviewed changes

test/jasmine/tests/plots_test.js

'geo',

'pie',

'ternary'

]);

Copy link

Collaborator

alexcjohnson Apr 12, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great test!

etpinard added status: reviewable and removed status: has TODOs labels Apr 12, 2018

typos in 📚

c7b81ec

etpinard mentioned this pull request Apr 13, 2018

Splom open items #2555

Open

7 tasks

etpinard and others added 6 commits April 13, 2018 14:44

Merge branch 'master' into splom-feature

8e9eaff

Update deps, adapt line2d code

ad5abb6

Fix select in splom large

c98071f

sub regl-splom -> regl-scattermatrix in comments

2d93469

Fix removing line passes

6c5a1e7

clear canvas on doTicksRelayout when hasOnlyLargeSploms

a7b08b3

This was referenced Apr 16, 2018

Finance refactor #2561

Merged

Implement better (more maintainable) way to redraw gl canvases on incremental updates #2562

Closed

etpinard merged commit 6f25ac6 into master Apr 16, 2018

etpinard deleted the splom-feature branch April 16, 2018 20:22

bcdunbar mentioned this pull request May 21, 2018

add splom docs plotly/documentation#930

Closed

3 tasks

etpinard mentioned this pull request Nov 9, 2018

Scattergl traces with multiple colors set sometimes incorrectly format points in other scattergl traces #3232

Closed

mschilli87 mentioned this pull request Nov 14, 2018

Matching axes #1549

Closed

alexcjohnson mentioned this pull request Apr 3, 2019

SPLOM diagonals plotly/plotly_express#42

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introducing splom traces #2505

Introducing splom traces #2505

etpinard commented Mar 28, 2018 •

edited

Loading

etpinard commented Mar 28, 2018 •

edited

Loading

alexcjohnson Mar 28, 2018

etpinard Mar 30, 2018 •

edited

Loading

alexcjohnson Mar 28, 2018

etpinard Mar 28, 2018

etpinard Mar 29, 2018

alexcjohnson Mar 28, 2018 •

edited

Loading

etpinard Mar 28, 2018

alexcjohnson Mar 28, 2018

amalinafz Jul 27, 2018

alexcjohnson Jul 27, 2018

amalinafz Jul 27, 2018

alexcjohnson Mar 28, 2018

alexcjohnson Mar 28, 2018

etpinard Mar 29, 2018

alexcjohnson Mar 28, 2018

alexcjohnson Mar 28, 2018

etpinard Mar 28, 2018

etpinard Mar 28, 2018 •

edited

Loading

etpinard Mar 29, 2018

etpinard Mar 29, 2018

etpinard Mar 28, 2018

etpinard Mar 28, 2018

etpinard Mar 28, 2018

etpinard commented Apr 12, 2018

alexcjohnson Apr 12, 2018

etpinard commented Apr 12, 2018 •

edited

Loading

alexcjohnson commented Apr 12, 2018

etpinard commented Apr 16, 2018

etpinard commented Apr 16, 2018

Introducing splom traces #2505

Introducing splom traces #2505

Conversation

etpinard commented Mar 28, 2018 • edited Loading

etpinard commented Mar 28, 2018 • edited Loading

things that work

things that do not work

Choose a reason for hiding this comment

etpinard Mar 30, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexcjohnson Mar 28, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

etpinard Mar 28, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

etpinard commented Apr 12, 2018

Choose a reason for hiding this comment

etpinard commented Apr 12, 2018 • edited Loading

alexcjohnson commented Apr 12, 2018

etpinard commented Apr 16, 2018

etpinard commented Apr 16, 2018

etpinard commented Mar 28, 2018 •

edited

Loading

etpinard commented Mar 28, 2018 •

edited

Loading

etpinard Mar 30, 2018 •

edited

Loading

alexcjohnson Mar 28, 2018 •

edited

Loading

etpinard Mar 28, 2018 •

edited

Loading

etpinard commented Apr 12, 2018 •

edited

Loading