New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow Overlay returned in DynamicMap to return varying number of items #1388

Closed
thoth291 opened this Issue Apr 30, 2017 · 11 comments

Comments

Projects
None yet
5 participants
@thoth291
Contributor

thoth291 commented Apr 30, 2017

In the notebook here I'm building a simple peakfinder widget.

So far 3 problems:

  1. Bokeh point size
    Here is what I get for the initial stage: You can clearly see the Blue point

screen shot 2017-04-29 at 9 34 28 pm

Then I move to the parameter for which that Blue point disappears.
screen shot 2017-04-29 at 9 34 51 pm

Then I move it back to the initial space - but the point size is reduced to the minimal
screen shot 2017-04-29 at 9 35 09 pm

  1. Matplotlib: point size is 100 and seems to be depend on fig_size - it's counterintuitive to me

  2. My calculation may take a while - how to delay the recalculation to the time when the slider is released?

P.S. I also want to update this DynamicMap to HoloMap so that when I save it goes to HTML, but when I wrap dmap with holomap - it does nothing... I'm sure I'm doing something wrong...

@thoth291

This comment has been minimized.

Contributor

thoth291 commented Apr 30, 2017

1 more issue.

  1. In the same notebook (scroll to the bottom) DynamicMap is not responding at all:

screen shot 2017-04-30 at 1 15 07 am

@thoth291

This comment has been minimized.

Contributor

thoth291 commented Apr 30, 2017

I think the issues 1. and 4. are connected.
In 4. apparently my lowess method was buggy and produced empty list which caused empty picks and etc. It is somehow affecting other methods to appear.
In 1. all of a sudden point is disappears and then reappears but with different style settings.

So if one of the plots in DynamicMap is changing something dramatic in one of the Overlays then it produces wrong image: with bad Style or Plot options...

This is example when 4. is not working

ldata=lowess[:, 0]
lxs=lowess[:, 1]

This is an example when 4. is working

ldata=lowess[:, 1]
lxs=lowess[:, 0]
@philippjfr

This comment has been minimized.

Member

philippjfr commented May 5, 2017

Sorry for being so slow to respond.

then I move it back to the initial space - but the point size is reduced to the minimal

Definitely seems like a bug I'll start investigating right away.

Matplotlib: point size is 100 and seems to be depend on fig_size - it's counterintuitive to me

This is an overall issue with sizing in matplotlib, the fig_size simply scales the fig_inches of the figure and leaves text and point sizes unaffected. I agree this is unintuitive and would like a scaling mechanism that scales everything in the figure.

how to delay the recalculation to the time when the slider is released?

Our widgets have some debouncing by default but disabling continuous updates seems like a good feature request.

I also want to update this DynamicMap to HoloMap so that when I save it goes to HTML, but when I wrap dmap with holomap - it does nothing... I'm sure I'm doing something wrong...

The way to achieve this is to explicitly sample the DynamicMap first like this:

hv.HoloMap(dmap[[0, 10, 20], [1, 2, 3]])

all of a sudden point is disappears and then reappears but with different style settings.

This does seem related to your first issue, I'll investigate.

@philippjfr philippjfr added the bug label May 6, 2017

@philippjfr philippjfr added this to the v1.8 milestone Jun 15, 2017

@philippjfr

This comment has been minimized.

Member

philippjfr commented Jun 17, 2017

Okay I've finally had a chance to go through your notebook. It's quite hard to follow complex examples like this and self-contained, minimal examples are much preferred when tracking down bugs. I've now tracked down the issue to the fact that your DynamicMap is returning an Overlay with varying numbers of Points. The number of items in an Overlay must be fixed when working with a DynamicMap because the plot cannot create new glyphs/artists on the fly. In bokeh you may use an NdOverlay with varying numbers of items but in matplotlib the number of Elements returned by a DynamicMap callable must remain fixed. I'm going to leave this issue open to better document these limitations.

@philippjfr philippjfr removed the bug label Jun 17, 2017

@philippjfr philippjfr removed this from the v1.8 milestone Jun 17, 2017

@philippjfr philippjfr added the docs label Jun 17, 2017

@philippjfr philippjfr changed the title from DynamicMap is not preserving Bokeh Point size to Document limitations surrounding Overlay and DynamicMap Jun 17, 2017

@jlstevens

This comment has been minimized.

Member

jlstevens commented Jun 17, 2017

Sounds like something we want to detail in the User Guide. Not sure which one though...

@thoth291

This comment has been minimized.

Contributor

thoth291 commented Jun 18, 2017

Thanks, @philippjfr. I'm sorry for the big notebook - will try to be better next time.
Yeah - inconsistency in-between the backends are very surprising and mostly unpredicted. It would be nice not to have them at all - but that's impossible to achieve...

Will you create a request on this issue which you mentioned above:

Our widgets have some debouncing by default but disabling continuous updates seems like a good feature request.

I'll switch to NdOverlay for bokeh, but will have to implement static plots using pure matplotlib. Which means that instead of reducing libraries from (bokeh, matplotlib) to (holoviews) - I add one more library dependency and have to keep 3 libraries (bokeh, matplotlib and holoviews) in mind while working with my data.

@stevievb

This comment has been minimized.

stevievb commented Jun 27, 2017

@philippjfr , I'm trying to use NdOverlay in bokeh with a varying number of items as you mentioned above but it doesn't seem to be working.

import holoviews as hv
import holoviews.plotting.bokeh # noqa (Activate backend)
from bokeh.layouts import column
from bokeh.plotting import curdoc
from pandas import DataFrame
from holoviews.streams import Stream
from bokeh.layouts import widgetbox
from bokeh.models.widgets import CheckboxGroup

doc = curdoc()

df1 = DataFrame({'x': [1, 2, 3], 'y': [2, 5, 8], 'desc': ['A', 'b', 'C']})
df2 = DataFrame({'x': [1.2, 2.2, 3.2], 'y': [2, 5, 8], 'desc': ['A', 'b', 'C']})
df3 = DataFrame({'x': [1.4, 2.4, 3.4], 'y': [2, 5, 8], 'desc': ['A', 'b', 'C']})

ShowMoreStream = Stream.define('Show More Stream', show_more=False)
show_more_stream = ShowMoreStream()

points1 = hv.Points(df1, kdims=['x', 'y'], vdims=['desc'])
points2 = hv.Points(df2, kdims=['x', 'y'], vdims=['desc'])
points3 = hv.Points(df3, kdims=['x', 'y'], vdims=['desc'])

renderer = hv.Store.renderers['bokeh'].instance(mode='server', holomap='server')
options = hv.Store.options(backend='bokeh')

def points_callback(show_more):
    if show_more:
        return hv.NdOverlay({'1': points1,
                             '2': points2,
                             '3': points3})
    else:
        return hv.NdOverlay(hv.NdOverlay({'1': points1,
                                        '2': points2}))

points_dmap = hv.DynamicMap(points_callback, streams=[show_more_stream])

def show_more_checkbox_change_callback(attr, old, new):
    if len(new):
        points_dmap.event(show_more=True)
    else:
        points_dmap.event(show_more=False)

checkbox_group = CheckboxGroup(labels=["Show More"])
checkbox_group.on_change("active", show_more_checkbox_change_callback)

plot = renderer.get_plot(points_dmap)

layout = column(plot.state, widgetbox(checkbox_group))

doc.add_root(layout)

When I hit the checkbox I get the message " OverlayPlot01231: Some Elements returned by the dynamic callback were not initialized correctly and could not be rendered."

If reverse the order so that the overlay with the most number of elements is displayed first I don't get and error message and I can toggle the element on and off but the legend does not update.

def points_callback(show_more):
    if show_more:
        return hv.NdOverlay(hv.NdOverlay({'1': points1,
                                        '2': points2}))
    else:
        return hv.NdOverlay({'1': points1,
                             '2': points2,
                             '3': points3})
@philippjfr

This comment has been minimized.

Member

philippjfr commented Jun 27, 2017

Thanks for the nice reproducible example, that message is unfortunately less informative than it could be, so I'll make sure to fix that and I'll have to document this stuff. Currently to be able to update NdOverlays easily we have something called "batched" mode, which collapses all the elements in your NdOverlay into a single ColumnDataSource. Batched mode doesn't support legends so we have decided that any NdOverlay with more than 10 items will be batched by default. You can reduce the batching limit with the legend_items plot option. In your example you can either reduce the legend_limit globally or for those NdOverlays in particular e.g.:

def points_callback(show_more):
    if show_more:
        items = {'1': points1, '2': points2})
    else:
        items = {'1': points1, '2': points2, '3': points3})
    return hv.NdOverlay(items).opts(plot=dict(legend_limit=0))

This is quite an annoyance so I'll make sure to document it. In future we can likely lift this restriction at least on bokeh server.

@shaginhekvs

This comment has been minimized.

shaginhekvs commented Jul 22, 2017

@philippjfr the problem partly still persists, if you flip the order such that

def points_callback(show_more):
    if not show_more:
        items = {'1': points1, '2': points2})
    else:
        items = {'1': points1, '2': points2, '3': points3})
    return hv.NdOverlay(items).opts(plot=dict(legend_limit=0))

you can only see the initially plotted 2 points now and the 3rd point cannot be enabled.

@philippjfr philippjfr changed the title from Document limitations surrounding Overlay and DynamicMap to Allow Overlay returned in DynamicMap to return varying number of items Nov 6, 2017

@philippjfr philippjfr added feature and removed docs labels Nov 6, 2017

@philippjfr

This comment has been minimized.

Member

philippjfr commented Nov 6, 2017

Since there is now a clear path forward toward allowing DynamicMaps to return Overlays with varying numbers of items I've repurposed the issue and assigned it to the next milestone.

@philippjfr philippjfr added this to the v1.10 milestone Nov 6, 2017

@philippjfr philippjfr self-assigned this Feb 9, 2018

@philippjfr philippjfr modified the milestones: v1.10, v1.11 Mar 19, 2018

@philippjfr

This comment has been minimized.

Member

philippjfr commented Mar 29, 2018

This is now supported with some caveats (namely in bokeh the toolbar will disappear but this is an issue that must be addressed in bokeh).

@philippjfr philippjfr closed this Mar 29, 2018

@philippjfr philippjfr modified the milestones: v1.11, v1.10 Mar 29, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment