Proposal: Ability to add new streams to DynamicMap.map()

[peterroelants]

This PR is a conceptual proposal and is not ready to merge. I would like to start a discussion on how to extend existing DynamicMaps with new streams.

I think the ability to extend DynamicMap with new streams would be a great addition. Sometimes when creating a complex dashboard I have plots that are derived from other plots. If the source plots are DynamicMaps I usually use DynamicMap.map() to create the derived plot. For example I might make a projection as a derived plot that can be added to a Adjoint layout. An example of such a projection could be:

x_proj = dmap_2d.map(
    lambda qm: hv.Curve(qm.dataset.transform(z=hv.dim("z").xr.sum(dim="y")),),
    specs=hv.QuadMesh
)

However, sometimes I might want to create an additional interaction with the derived plot. However, adding a new stream with events, and the required effect of the stream event on the data, to an existing DynamicMap is currently not possible as far as I know (please correct me if I'm wrong).

I was thinking I would be nice to extend DynamicMap.map (or maybe create a new mapping method) to allow for new streams to be passed to .map(streams=...), of which the parameters can be used in the map_fn function.

As a rough illustration how this could work I created this PR. And have created the following example that illustrates creating a derived plot (histogram) that depends on a new stream (hv.streams.Selection1D)

import numpy as np
import holoviews as hv
import panel as pn

hv.extension('bokeh')

# Widgets
x_offset_slider = pn.widgets.FloatSlider(
    name='x-offset', 
    start=-5, end=5, step=0.1, value=0.
)
y_offset_slider = pn.widgets.FloatSlider(
    name='y-offset', 
    start=-5, end=5, step=0.1, value=0.
)
widget_box = pn.WidgetBox(
    x_offset_slider,
    y_offset_slider
)

widget_streams = hv.streams.Params.from_params(dict(
    x_offset=x_offset_slider.param.value, 
    y_offset=y_offset_slider.param.value
))



def gen_points_overlay(x_offset, y_offset):
    return hv.Points(np.random.randn(1000, 2) + np.array([x_offset, y_offset]), kdims=['x', 'y'], label='p1')

dmap_2d = hv.DynamicMap(
    gen_points_overlay, streams=[*widget_streams]
).opts(
    hv.opts.Points(framewise=True, tools=['box_select', 'lasso_select'])
)

selection = hv.streams.Selection1D(source=dmap_2d)

# Map DynamicMap with indexes from additional selection stream
def hist_mapper(points: hv.Points, index, **kwargs):
    selected_points = points
    if index:
        selected_points = points.iloc[index]
    return hv.operation.histogram(selected_points, dimension='y')

dmap_hist = dmap_2d.map(map_fn=hist_mapper, specs=hv.Points, streams=[selection])

plot = dmap_2d + dmap_hist

app = pn.Column(plot, widget_box)
display(app)

Using the histogram and the selection are just examples of a derived plot and an additional stream here. They could be any other derivation on the original plot's data with any other event stream.

I would love to know what y'all think. Is this something Holoviews want to support? Do you think extending DynamicMap.map(streams=...) is the right way, or are there other routes to implementing this behaviour?

[philippjfr]

This is a great idea overall, but as far as I can tell it's an idea we have already fully implemented as part of the .apply method. Your example can already be written as:

dmap_hist = dmap_2d.apply(hist_mapper, index=selection.param.index)

or

dmap_hist = dmap_2d.apply(hist_mapper, streams=[selection])

I have no particular objections to supporting streams and parameters on .map as well, but we explicitly created apply so you could easily .apply functions dynamically without having to declare explicit specs, i.e. it automatically applies to ViewableElement types which would either be an Element or an (Nd)Overlay.

[peterroelants]

Thank you for pointing out .apply(), I overlooked it's capability of doing this, it is indeed what I'm looking for.

I can conform that both:

dmap_hist = dmap_2d.apply(apply_function=hist_mapper, streams=[selection])

and

dmap_hist = dmap_2d.apply(apply_function=hist_mapper, index=selection.param.index)

work.

Full example:

import numpy as np
import holoviews as hv
import panel as pn

hv.extension('bokeh')

# Widgets
x_offset_slider = pn.widgets.FloatSlider(
    name='x-offset', 
    start=-5, end=5, step=0.1, value=0.
)
y_offset_slider = pn.widgets.FloatSlider(
    name='y-offset', 
    start=-5, end=5, step=0.1, value=0.
)
widget_box = pn.WidgetBox(
    x_offset_slider,
    y_offset_slider
)

widget_streams = hv.streams.Params.from_params(dict(
    x_offset=x_offset_slider.param.value, 
    y_offset=y_offset_slider.param.value
))



def gen_points_overlay(x_offset, y_offset):
    return hv.Points(np.random.randn(1000, 2) + np.array([x_offset, y_offset]), kdims=['x', 'y'], label='p1')

dmap_2d = hv.DynamicMap(
    gen_points_overlay, streams=[*widget_streams]
).opts(
    hv.opts.Points(framewise=True, tools=['box_select', 'lasso_select'])
)

selection = hv.streams.Selection1D(source=dmap_2d)

# Map DynamicMap with indexes from additional selection stream
def hist_mapper(points: hv.Points, index):
    selected_points = points
    if index:
        selected_points = points.iloc[index]
    return hv.operation.histogram(selected_points, dimension='y')

# dmap_hist = dmap_2d.apply(apply_function=hist_mapper, streams=[selection])
dmap_hist = dmap_2d.apply(apply_function=hist_mapper, index=selection.param.index)

plot = dmap_2d + dmap_hist

app = pn.Column(plot, widget_box)
display(app)

Thanks! I will be closing this proposal.

EDIT 2021-08-06: .apply() does not support the specs argument. I didn't notice because of the **kwargs argument in hist_mapper. I updated the example accordingly.

[peterroelants]

I noticed one difference between map and apply:
.apply() does not have a specs argument to match only certain elements in the DynamicMap.

[jbednar]

That would be a nice addition!