Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve DataTable performance #8227

Closed
wabu opened this Issue Sep 8, 2018 · 11 comments

Comments

3 participants
@wabu
Copy link

commented Sep 8, 2018

Performance with bokeh models containing data-table widgets is slow when the model gets updated (streaming data to it or changing widgets), adding up to some seconds of delay. When debugging it in the browser, Chromium warns about "Recalculate Style
Warning: Forced reflow is a likely performance bottleneck." (see screenshot below).

bokeh 0. 13.0
python 3.6.6 Anaconda
Chromium 66.0.3359.139
Debian 4.16.5-1
Linux 4.16.0-1-amd64

Here's an example, where adding a button cases the delay, but also had the problem when adding data to a plot with holoviews pipes

import bokeh.io as bki
import bokeh.models as bkm
import bokeh.layouts as bkl

doc = bki.curdoc()

from datetime import date
from random import randint, choice

n = 20
data = dict(
        dates=[date(2014, 3, i%30+1) for i in range(n)],
        downloads=[randint(0, 100) for i in range(n)],
        foo=[randint(0, 100) for i in range(n)],
        bar=[randint(0, 100) for i in range(n)],
        baz=[randint(0, 100) for i in range(n)],
    )
source = bkm.ColumnDataSource(data)

columns = [
        bkm.TableColumn(field="dates", title="Date", formatter=bkm.DateFormatter()),
        bkm.TableColumn(field="downloads", title="Downloads"),
        bkm.TableColumn(field="foo", title="Foo"),
        bkm.TableColumn(field="bar", title="Bar"),
        bkm.TableColumn(field="baz", title="Baz"),
    ]

table1 = bkm.DataTable(source=source, columns=columns[:-1])#, editable=True)
table2 = bkm.DataTable(source=source, columns=columns[1:])#, editable=True)

action = bkm.Button(label="click me")
foo = bkm.Button(label="foo")
box = bkl.widgetbox([action, foo], sizing_mode='scale_width')

def act():
    #last = label = box.children[1].label
    #while label == last:
    label = choice(["foo", "bar", "baz", "bla"])
    foo = bkm.Button(label=label)
    box.children.insert(1, foo)
                    
action.on_click(act)

doc.add_root(bkl.column(box, bkl.widgetbox(table1, table2)))

2018-09-08-112531_1364x719_scrot

@wabu

This comment has been minimized.

Copy link
Author

commented Sep 9, 2018

Here are some detailed js call trees with calls that generate performance warnings:

0 ms0 % | 39.5 ms100.0 % | Function Call |
-- | -- | -- | --
0 ms0 % | 39.5 ms100.0 % | i.socket.onmessageconnection.js:61 |
0 ms0 % | 39.5 ms100.0 % | t._on_messageconnection.js:189 |
0 ms0 % | 39.5 ms100.0 % | ACK.t.msgtype._current_handlerconnection.js:242 |
0 ms0 % | 39.5 ms100.0 % | t._steady_state_handlerconnection.js:255 |
0 ms0 % | 39.5 ms100.0 % | t.handlesession.js:16 |
0 ms0 % | 39.5 ms100.0 % | t._handle_patchsession.js:69 |
0 ms0 % | 39.5 ms100.0 % | P.apply_json_patchdocument.js:674 |
0 ms0 % | 39.5 ms100.0 % | u.setvhas_props.js:222 |
0 ms0 % | 39.5 ms100.0 % | u._setvhas_props.js:183 |
0 ms0 % | 39.5 ms100.0 % | e.emitsignaling.js:67 |
0 ms0 % | 39.5 ms100.0 % | t.emitsignaling.js:50 |
0 ms0 % | 39.5 ms100.0 % | (anonymous)widget_box.js:14 |
0 ms0 % | 39.5 ms100.0 % | t.rebuild_child_viewslayout_dom.js:196 |
0 ms0 % | 39.5 ms100.0 % | t.layoutlayout_dom.js:156 |
0 ms0 % | 39.5 ms100.0 % | t.layoutlayout_dom.js:156 |
0 ms0 % | 39.5 ms100.0 % | t._do_layoutlayout_dom.js:162 |
0 ms0 % | 39.5 ms100.0 % | t._layoutlayout_dom.js:182 |
0 ms0 % | 39.5 ms100.0 % | t._layoutlayout_dom.js:182 |
0 ms0 % | 39.5 ms100.0 % | t._layoutlayout_dom.js:182
0 ms0 % | 39.5 ms100.0 % | e.renderdata_table.js:160 |
0 ms0 % | 39.5 ms100.0 % | SlickGridslick.grid.js:35 |
0 ms0 % | 39.5 ms100.0 % | initslick.grid.js:185 |
0 ms0 % | 39.5 ms100.0 % | finishInitializationslick.grid.js:279 |
0 ms0 % | 39.5 ms100.0 % | bindAncestorScrollEventsslick.grid.js:500 |
39.5 ms100.0 % | 39.5 ms100.0 % | Recalculate Styleslick.grid.js:504

0 ms0 % | 39.5 ms100.0 % | Function Call |
-- | -- | -- | --
0 ms0 % | 39.5 ms100.0 % | i.socket.onmessageconnection.js:61 |
0 ms0 % | 39.5 ms100.0 % | t._on_messageconnection.js:189 |
0 ms0 % | 39.5 ms100.0 % | ACK.t.msgtype._current_handlerconnection.js:242 |
0 ms0 % | 39.5 ms100.0 % | t._steady_state_handlerconnection.js:255 |
0 ms0 % | 39.5 ms100.0 % | t.handlesession.js:16 |
0 ms0 % | 39.5 ms100.0 % | t._handle_patchsession.js:69 |
0 ms0 % | 39.5 ms100.0 % | P.apply_json_patchdocument.js:674 |
0 ms0 % | 39.5 ms100.0 % | u.setvhas_props.js:222 |
0 ms0 % | 39.5 ms100.0 % | u._setvhas_props.js:183 |
0 ms0 % | 39.5 ms100.0 % | e.emitsignaling.js:67 |
0 ms0 % | 39.5 ms100.0 % | t.emitsignaling.js:50 |
0 ms0 % | 39.5 ms100.0 % | (anonymous)widget_box.js:14 |
0 ms0 % | 39.5 ms100.0 % | t.rebuild_child_viewslayout_dom.js:196 |
0 ms0 % | 39.5 ms100.0 % | t.layoutlayout_dom.js:156 |
0 ms0 % | 39.5 ms100.0 % | t.layoutlayout_dom.js:156 |
0 ms0 % | 39.5 ms100.0 % | t._do_layoutlayout_dom.js:162 |
0 ms0 % | 39.5 ms100.0 % | t._layoutlayout_dom.js:182 |
0 ms0 % | 39.5 ms100.0 % | t._layoutlayout_dom.js:182 |
0 ms0 % | 39.5 ms100.0 % | t._layoutlayout_dom.js:182
0 ms0 % | 39.5 ms100.0 % | e.renderdata_table.js:160 |
0 ms0 % | 39.5 ms100.0 % | SlickGridslick.grid.js:35 |
0 ms0 % | 39.5 ms100.0 % | initslick.grid.js:185 |
0 ms0 % | 39.5 ms100.0 % | finishInitializationslick.grid.js:279 |
0 ms0 % | 39.5 ms100.0 % | bindAncestorScrollEventsslick.grid.js:500 |
5.1 ms100.0 % | 5.1 ms100.0 % | Layoutslick.grid.js:504

0 ms0 % | 34.7 ms100.0 % | Function Call |
-- | -- | -- | --
0 ms0 % | 34.7 ms100.0 % | i.socket.onmessageconnection.js:61 |
0 ms0 % | 34.7 ms100.0 % | t._on_messageconnection.js:189 |
0 ms0 % | 34.7 ms100.0 % | ACK.t.msgtype._current_handlerconnection.js:242 |
0 ms0 % | 34.7 ms100.0 % | t._steady_state_handlerconnection.js:255 |
0 ms0 % | 34.7 ms100.0 % | t.handlesession.js:16 |
0 ms0 % | 34.7 ms100.0 % | t._handle_patchsession.js:69 |
0 ms0 % | 34.7 ms100.0 % | P.apply_json_patchdocument.js:674 |
0 ms0 % | 34.7 ms100.0 % | u.setvhas_props.js:222 |
0 ms0 % | 34.7 ms100.0 % | u._setvhas_props.js:183 |
0 ms0 % | 34.7 ms100.0 % | e.emitsignaling.js:67 |
0 ms0 % | 34.7 ms100.0 % | t.emitsignaling.js:50 |
0 ms0 % | 34.7 ms100.0 % | (anonymous)widget_box.js:14 |
0 ms0 % | 34.7 ms100.0 % | t.rebuild_child_viewslayout_dom.js:196 |
0 ms0 % | 34.7 ms100.0 % | t.layoutlayout_dom.js:156 |
0 ms0 % | 34.7 ms100.0 % | t.layoutlayout_dom.js:156 |
0 ms0 % | 34.7 ms100.0 % | t._do_layoutlayout_dom.js:162 |
0 ms0 % | 34.7 ms100.0 % | t._layoutlayout_dom.js:182 |
0 ms0 % | 34.7 ms100.0 % | t._layoutlayout_dom.js:182 |
0 ms0 % | 34.7 ms100.0 % | t._layoutlayout_dom.js:182 |
0 ms0 % | 34.7 ms100.0 % | e.renderdata_table.js:160 |
0 ms0 % | 34.7 ms100.0 % | SlickGridslick.grid.js:35 |
0 ms0 % | 34.7 ms100.0 % | initslick.grid.js:185 |
0 ms0 % | 34.7 ms100.0 % | finishInitializationslick.grid.js:279 |
0 ms0 % | 34.7 ms100.0 % | resizeCanvasslick.grid.js:1709 |
0 ms0 % | 34.7 ms100.0 % | getViewportHeightslick.grid.js:1699 |
0 ms0 % | 34.7 ms100.0 % | cssjquery.js:5047 |
0 ms0 % | 34.7 ms100.0 % | getjquery.js:5079 |
0 ms0 % | 34.7 ms100.0 % | cssjquery.js:5047 |
0 ms0 % | 34.7 ms100.0 % | jebokeh-tables.min.js?v=204673f…:0 |
0 ms0 % | 34.7 ms100.0 % | getPropertyValue |
34.7 ms100.0 % | 34.7 ms100.0 % | Recalculate Stylejquery.js:4811

As I am only familiar with python, I can't get into the details of the issue myself.

@wabu

This comment has been minimized.

Copy link
Author

commented Sep 10, 2018

Inside the real-world use-case, adding 6 tables restricting the rows to only a single one will already add a delay of around 2 seconds, doubling when using full tables (20-50 rows).

@mattpap

This comment has been minimized.

Copy link
Contributor

commented Sep 10, 2018

There are two issues here. One is that currently data tables (and everything else) are excessively rendered. This is fixed in PR #8085. The other thing is that changing layout is an expensive operation on its own. However, you are changing a branch that's unrelated to data tables, so they shouldn't be affected. I will need to assert in #8085 that that's actually true.

@mattpap mattpap added this to the 1.0 milestone Sep 10, 2018

@mattpap mattpap added this to In progress in Focus: Layout Sep 10, 2018

@wabu

This comment has been minimized.

Copy link
Author

commented Sep 10, 2018

Thanks for the follow up. I'll keep an eye on what's happening on #8085.

@mattpap mattpap moved this from In progress to Implemented in Focus: Layout Sep 12, 2018

@bryevdv bryevdv modified the milestones: 1.0, 1.1 Oct 1, 2018

@bryevdv

This comment has been minimized.

Copy link
Member

commented Oct 9, 2018

Hrm, on my current data table PR the buttons are added immediately with no delay, but it does not affect any layout things. I have not tried with master or the latest release to compare though

@wabu

This comment has been minimized.

Copy link
Author

commented Oct 11, 2018

I'll check it with #8085

@bryevdv

This comment has been minimized.

Copy link
Member

commented Oct 11, 2018

@wabu just to be clear, I was unable to reproduce a delay on master, which has not yet merged #8085

@wabu

This comment has been minimized.

Copy link
Author

commented Oct 12, 2018

I'll have a look, thanks for trying!

@wabu

This comment has been minimized.

Copy link
Author

commented Oct 18, 2018

Tested with fresh build of master and still having problems. Clicking the button something like 10-times in a row, the browser will use all CPU and will take some seconds to catch up.

Same happens with a bigger dashboard when streaming data into a bokeh plot with holoviews.

@wabu

This comment has been minimized.

Copy link
Author

commented Oct 18, 2018

checked it on #8085, this one helps with the performance allot, but some of the layouting of bigger dashboard breaks.

@bryevdv

This comment has been minimized.

Copy link
Member

commented Jan 31, 2019

checked with current iteration of #8085, pressed the butting ~30 times and every button added instantly

@bryevdv bryevdv closed this in #8085 Feb 1, 2019

@mattpap mattpap moved this from Implemented to Done in Focus: Layout Mar 2, 2019

@bryevdv bryevdv changed the title DataTable performance Improve DataTable performance Mar 24, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.