Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Browser rendering extremely slow when many figures in a gridplot #6294

Closed
Boubsi opened this Issue May 16, 2017 · 23 comments

Comments

@Boubsi
Copy link

commented May 16, 2017

Hi all,

when I have many figures in a gridplot, browsers take a very very long time to render the html file, even if the data is very small.
For instance, when I run

import bokeh.plotting as bplt
import bokeh.io as bio
n_rows = 10
n_columns = 6
data_x = [1, 2, 3]
data_y = [1, 2, 3]
grid = []
for __ in range(n_rows):
    row = []
    for __ in range(n_columns):
        sub_fig = bplt.figure(width=200, height=100)
        sub_fig.line(data_x, data_y)
        row.append(sub_fig)
    grid.append(row)

fig = bplt.gridplot(grid)

bplt.reset_output()
bio.output_file('test.html')
bplt.save(fig)

and then open the 'test.html' file in a browser (firefox 53.0.2 or chromium 58.0.3029.96, on ubuntu 16.04 LTS), it takes 40 seconds to render, with 100 % use of one of my CPU (cf below)
cpu_bokeh

Moreover, when the html file is open, the memory used increases by 400 Mo, even if the html file only weights 380 ko.

Do you know if there is a workaround for this problem? Maybe, this slowness is due to one special feature of bokeh, I could bypass?
Otherwise, is there a chance that bokeh will soon optimize the rendering of multiple plots?
I would love to use bokeh in my project, but I need to make this kind of plots and the rendering time is two long.

Thank you for your work!

EDIT: bokeh==0.12.5

@Boubsi Boubsi changed the title Browser rendring extremly slow when many figures in a gridplot Browser rendering extremely slow when many figures in a gridplot May 16, 2017

@mattpap

This comment has been minimized.

Copy link
Contributor

commented May 16, 2017

@Boubsi, which version of bokeh?

@bryevdv

This comment has been minimized.

Copy link
Member

commented May 16, 2017

@Boubsi if you need 60 static plots, it's possible bokeh is not the right tool for you. The features for interactivity, streaming, layout, etc are not free, they come with trade-offs. Generally speaking, Bokeh is better with fewer plots, that each have more data, than it is with very many plots with a little data. It's possible there are optimizations we can make to improve things in some case but I expect this general statement will always be true to a degree.

That said, the next release should have options for PNG and SVG export so perhaps Bokeh would be useful for generating static images ahead of time, that you can embed as images.

@Boubsi

This comment has been minimized.

Copy link
Author

commented May 16, 2017

@mattpap: bokeh==0.12.5
@bryevdv: thank for the info. Indeed, maybe I will have to render my html files with matplolib + mpl3d (there is not such low rendering problems when many subplots are in stake), but the possibility to make a bokeh server would have been a great feature for my project in future...
A compromise would be to find a solution for the plots to be rendered successively instead of waiting that all plots are genereted before rending the all page (from the user point of view, it could be acceptable): do you think I would be possible to do that? Maybe by cutting my figure as several subfigures, generate html for each one of them and embed them some wrapper?

@bryevdv

This comment has been minimized.

Copy link
Member

commented May 16, 2017

@Boubsi templating individual app elements individually is still an open feature request: #4986

There are other ways of splitting things up, and they might possibly be helpful, but it's hard to say anything specific without more details. However, this is venturing into the territory of "general support question" so I'd ask that the discussion continue on the mailing list, rather than the issue tracker:

https://groups.google.com/a/continuum.io/forum/#!forum/bokeh

@David-Herman

This comment has been minimized.

Copy link
Contributor

commented May 17, 2017

@bryevdv Thank you for your response.

I am also working with Bokeh for use in a dashboard and can end up with 12-40 plots. So the use case of many plots is not uncommon. In my case specifically, I am using Tabs() to group the plots into groups. Perhaps modifying the default behavior to render only upon tab click per each tab would be helpful to decrease the initial page load time but still allow a user to serve additional plots per a single page load.

@Boubsi

This comment has been minimized.

Copy link
Author

commented May 17, 2017

@David-Herman
I want to try something like what you're suggesting, by using a bokeh server and RadioButtonGroups. For instance, when a RadioButton is pushed, the data of the glyph's figure are replaced. Il hope it will allow me to reduce the number of figures, thus decreasing the initial page load.

@David-Herman

This comment has been minimized.

Copy link
Contributor

commented May 17, 2017

@Boubsi

In my case I am serving the content via Django. So I am outputting the div and script via components and then embedding them via the Django template for rendering into HTML. I am considering using Bootstrap tabs in place of Bokeh and the wrapping the script within another script that runs on button press. I haven't yet prototyped this yet.

@jmc416

This comment has been minimized.

Copy link

commented Oct 23, 2017

bokeh==0.12.6
I have a similar use-case, and see slow loading times

@bryevdv

This comment has been minimized.

Copy link
Member

commented Oct 23, 2017

@jmc416 as I stated above, Bokeh cannot satisfy every use case. It has made purposeful tradeoffs to cater towards very high interactivity. If you need 60 plots on a page ("similar to the OP") then Bokeh is probably not the right tool for your scenario.

@jouellnyc

This comment has been minimized.

Copy link

commented Oct 27, 2017

At the risk of muddying the waters, I had the same issue but w/only 6 plots. At first it took about 1-3 second and then, it took about 90+ seconds (it did finish) to render what was once quick plotting.

I didn't change anything in terms of the plotting BUT started to integrate pandas and other things in my script and probably fussed w/the packages.

I really think my python packages got inconsistent (I had Ubuntu managed python3.x as well as an anaconda install). As I'm somewhat new to python so was just happy it was working paid no mind to the proper setup until it got slow.

I only sorted out my issue after trying to sort out my packages when I also noticed I needed to do:

from bokeh.layouts import gridplot
instead of
from bokeh.io import gridplot

because I was also getting: ImportError: cannot import name 'gridplot' all of a sudden.

Probably my error but my (6 plots page) are super fast now (actually they seem even faster than when I started!)

Currently using 0.12.10 bokeh w/Spyder 3.2.4(Ananconda w/python 3.6) on Ubuntu 16.04.1 LTS works great as well as the OS supplied python (which happens to be 3.5)

I realize the report is anecdotal as I didn't take detail notes as I was fumbling with my setup but I hope this may help.

@jmc416

This comment has been minimized.

Copy link

commented Nov 3, 2017

Thanks for your responses. I just wanted to mention that switching to a more powerful graphics card (GeForce GTX 960M from Intel skylake integrated graphics) reduces the load time from about >10s to <1s.

@synapticarbors

This comment has been minimized.

Copy link

commented Nov 8, 2017

Just ran into the same issue plotting the evolution of a bunch of parameters for an MCMC run. My use case involves line plots with 10-60 subplots, with 1-10 lines per plot, each with 100-1000 data points. I'm seeing load times of close to 60 seconds using Bokeh 0.12.4.

Bokeh made it so easy to generate a really useful and beautiful visualization. I'm not sure what can be done about the performance piece, but wanted to add my voice to the crowd that would find it really useful to improve performance in this regime.

@mattpap

This comment has been minimized.

Copy link
Contributor

commented Nov 8, 2017

Bokeh 0.12.4

This is almost a year old version of bokeh. For a discussion about performance this needs to be the most recent one. A lot can change in a month and in a year we are speaking about quite different piece of software. Though, as other posts indicate, not that much changed with regards to multi-plot performance.

@mattpap mattpap added type: task and removed type: discussion labels Nov 8, 2017

@mattpap mattpap self-assigned this Nov 8, 2017

@mattpap mattpap added this to the 0.12.x milestone Nov 8, 2017

@mattpap

This comment has been minimized.

Copy link
Contributor

commented Nov 14, 2017

So far we were able to reduce the rendering time of the original example by half (more or less). From here there is much room for improvement across bokehjs, and especially in layout subsystem, which currently dominates initialization time of a multiple plot system with small data size.

@intijk

This comment has been minimized.

Copy link

commented May 2, 2018

I have a image which is quite small, with RGB channel

croppedImage.shape
(30, 29, 3)

I use below code to draw it

p = figure(plot_width=400, plot_height=400, match_aspect=True, output_backend="webgl")

for i in range(croppedImage.shape[0]):
    #for j in range(croppedImage.shape[1]):
    for j in range(29):
        c=tuple((croppedImage[i][j]*255).astype(int).tolist())
        p.rect(j,croppedImage.shape[0]-i,width=1, height=1, color=c)
show(p)

It would take more than 3 min to render the result. When p.rect() line is commented, it return instantly.

@mattpap

This comment has been minimized.

Copy link
Contributor

commented May 2, 2018

@intijk, this is a different issue, and bokeh isn't optimized for this, but for having a single (or few) p.rect() (or other glyphs) with vectorized inputs. Additionally, webgl backend may heavily influence initialization time when a lot of glyphs are being used (see #4888).

@bryevdv

This comment has been minimized.

Copy link
Member

commented May 2, 2018

@intijk to be more explicit, Bokeh glyphs are vectorized. It is always preferable and more performant e.g. to have one call to rect with a 100 rows of data (to draw 100 rects) than the converse (i.e. 100 calls to rect, that each draw only 1 rect). This is an explicit design choice and is unlikely to change.

@intijk

This comment has been minimized.

Copy link

commented May 2, 2018

@bryevdv then how to give those rect different colors?

@bryevdv

This comment has been minimized.

Copy link
Member

commented May 2, 2018

@intijk please take support questions outside the tracker (i.e. to the public mailing list, or SO, or gitter) and also please refer to the docs as vectorizing colors is one of the most fundamental things that is done in almost every published example e.g.

https://bokeh.pydata.org/en/latest/docs/user_guide/quickstart.html#more-examples

@intijk

This comment has been minimized.

Copy link

commented May 2, 2018

Sorry for post to the wrong place, but anyway, thank you for the vectorization suggestion. It improved the performance a lot.

@bryevdv bryevdv modified the milestones: 0.13.x, short-term Sep 11, 2018

@bryevdv bryevdv modified the milestones: 0.13.x, short-term Sep 11, 2018

@gblack686

This comment has been minimized.

Copy link

commented Sep 25, 2018

Just started using Bokeh and love it. I'm experiencing these same load time issues and wondering if the plot points (glyphs) are the main reason for the lag or if other parts of the code will inhibit performance at similar rates.

@mattpap mattpap modified the milestones: short-term, 1.0 Sep 25, 2018

@mattpap mattpap added this to Implemented in Focus: Layout Sep 25, 2018

@mattpap mattpap moved this from In Progress to Implemented in Focus: Performance and Optimization Sep 25, 2018

@bryevdv

This comment has been minimized.

Copy link
Member

commented Sep 25, 2018

@gblack686 one reason for slow performance with many figures was that we had been using a constraint based solver for computing overall layout. While it was an attractive idea for several reasons, it also had serious drawbacks. It is being removed in #8085 and replaced with a much simpler and faster layout system. So basically: stay tuned for upcoming developments.

@bryevdv bryevdv modified the milestones: 1.0, 1.1 Oct 1, 2018

@bryevdv

This comment has been minimized.

Copy link
Member

commented Jan 31, 2019

I upped this to 150 plots with #8085 and it ran and rendered in about 3 seconds on my macbook

@bryevdv bryevdv closed this in #8085 Feb 1, 2019

@mattpap mattpap moved this from Implemented to Done in Focus: Layout Mar 2, 2019

@mattpap mattpap moved this from Implemented to Done in Focus: Performance and Optimization Mar 2, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.