Getting values of selected rows from a DataTable after reordering with sort

[jms90h5]

I'm trying to find a way to query the values on selected rows of a DataTable after it has been sorted by clicking on one of the column headers.

Before the table row order gets modified by the sort operation I could use the selected indexes available in a CustomJS callback to get the values contained in the selected rows from the ColumnDataSource used as input to the table. However, when the table rows are reordered with a sort those selection indexes relate to the new order of the rows that are displayed. (i.e. the order oder of the source stays the same but the display order is different.) The reported selection indexes correctly match the display order (e.g. if the third row is selected a selection index of 2 will be reported). However, I can't figure out how to get the values that are displayed on that row since it now isn't the third row of the input data source.

---

Bryan Van de ven commented:

Looking at the current code:

https://github.com/bokeh/bokeh/blob/master/bokehjs/src/coffee/models/widgets/data_table.coffee#L52-L86

It looks like it actually is trying to update the data source order. I'm not sure how I feel about that. I think maybe I hope that doesn't work, I think a better approach would be to maintain an explicit sorter to use to map to the original data.

---

My initial reaction is that I agree with Bryan that actually reordering the data source might not be a great idea. However a bit more investigation shows that the data source is not actually being reordered. Either that or the reordered one is not getting exposed to either the javascript in the callback or python executing in a jupyter notebook.

I should add that I've been testing this on v.11RC2 but will try with the released one also.

[dennisobrien]

This sounds like the same issue I ran into and addressed with this PR: #2925

But I see from the bug description that you experience the problem in 0.11RC2. So maybe the issue is something different? Can you include a minimal code snippet to demonstrate this?

thanks,
Dennis

[jms90h5]

Hi Dennis,

Although I wouldn't be surprised if this issue and PR: #2925 related to the same parts of the code, I suspect this is actually a different problem. I've attached a short notebook that can be used to demonstrate the problem with determining the values contained on the selected rows.

If there is some workaround or different way of accessing the selected rows after they have been sorted it would be very helpful. As it stands I just have to tell my users that they can either pick things in the table for subsequent display OR sort the table to look at it differently, but not both :(

sorted_table_test.ipynb.zip

[dennisobrien]

Hi @jms90h5

Sorry I missed the notification on your response.

I see the problem you are talking about. It looks like the selection indices are not being updated on a sort. Or, the code uses the layout indices (which do not get sorted) instead of the data indices (which do get sorted).

I'm not sure if this bug is in slickgrid or in Bokeh's data_table.coffee
https://github.com/bokeh/bokeh/blob/32922a07b0cfdceeb680bab5964294a6b6bf0ec0/bokehjs/src/coffee/models/widgets/data_table.coffee
My guess is the same thing happens with a filtered DataTable.

Some things to look into:

• w.r.t. slickgrid, is the client responsible for mainaining selections between sorts, filters, paginations, etc., or is this something slickgrid handles? Or, is this a bug in slickgrid?
• If this is a bug in bokeh (probably in data_table.coffee), is the code indexing incorrectly? Does the sort callback need to explicitly handle selections?

My coffeescript fluency is very limited, so it might be best to get some advice from the contributors to data_table.coffee. @bryevdv and @canavandl -- If you have any thoughts on this that would be most helpful.

BTW, I had no idea you could get a reference to the IPython kernel from the JavaScript callback! That is a neat trick. You should put that in a gist to share.

cheers,
Dennis

[chanansh]

Is there a solution for this problem? How do I know which line was selected after sorting? I need this urgently. Thanks!

[bryevdv]

This probably should have been promoted from discussion a long time ago, but there are alot of issues and sometimes things get lost. I have added this to the 0.12.6 milestone, but that will be a few months off at least. I wish I had something better to report at this time.

[nicain]

I just hit this as well. In my hands, it looks like it is not possible to get selected data (i.e. rows) from the data table back in python. This is because the "selected" property of the DataTable refers to the row on the figure, and not the row in the source ColumnDataSource. These are the same before sorting, but sorting breaks the relationship.

Unless I am missing something, this makes sortable, selectable DataTables unusable. Am I wrong here?

[bryevdv]

Unless I am missing something, this makes sortable, selectable DataTables unusable. Am I wrong here?

I'm not sure I'd disagree. Basically the table needs to maintain a "sorter" array that describes the current permutation from the original data order. Then when selections happen on the table, the table row index is mapped to the appropriate CDS index using the sorter. Conversely when a selection happens on a CDS, the same process is used in reverse to highlight the correct rows.

There is a question about getting the literal row indexes of the table when there is a selection. I think this is not useful? Without knowing how a sort permuted from the original order (that is in the CDS) this information does not really have any purpose. If that's wrong, I think it would need to be a property on the table (not the CDS) and I think it would need to be read-only a well. Being settable from python would not be semantically consistent.

Also possibly related: #1700

[nicain]

Basically the table needs to maintain a "sorter" array that describes the current permutation from the original data order.

I agree; if selection on the CDS via whatever route (other widget, programmatic, etc) and selection via the DT is to be supported, this data structure seem necessary and sufficient to do the assignment.

There is a question about getting the literal row indexes of the table when there is a selection. I think this is not useful?

I agree completely

I also came across https://github.com/bokeh/bokeh/wiki/Filterable-Data-Source while trying to understand the issue to build a short-term work around; source2 in the example is playing the role of a "selector" array. Perhaps a something similar could be the sorter array necessary to do the mapping? The mapping is provided by the "#" column (the first one in the DT, seems always on), but I don't know how to access it in the JS callback. Having a Filterable (and sortable) Data Source to turn on sorted subsets of glyphs would be amazing!

Thanks for all your work!

[bryevdv]

@jms90h5 @dennisobrien @nicain @chanansh I have the start of a fix for this in the linked PR #6180 I'd like to continue working on this PR all during the 0.12.6 dev cycle, just making data table generally more solid. If you have the ability to build and test from source I would definitely appreciate help with testing out this code in real-world use-cases.