DynamicMap with NdOverlays of variable length

[jlstevens]

I would expect to be able to change the number of items in an NdOverlay returned from a d=DynamicMap. The following example controls the number of items in the NdOverlay via a stream.

Shuffling the items works as the number of items stays the same. Increasing the number of items (e.g selected.event(items=list(range(NUM+1))) at the end) results in incorrect output and reducing the number of items (e.g selected.event(items=list(range(NUM-1)))causes the traceback detailed below:

Screenshot

Code

import random
import holoviews as hv
hv.extension('bokeh')

NUM=20
selected = hv.streams.Stream.define('selected', items=list(range(NUM)))()

def make_overlay(pos, count): 
    delta = 0.5/count
    bottom, top  = delta * pos, delta * (pos+1)
    data = [el for el in range(100) if (el % (pos+1)==0)]
    path = hv.Path([[(sample,bottom),(sample,top)]  for sample in data], vdims=[])
    return hv.Overlay([path])

def callback(items):
    paths = []
    for i,event_type in enumerate(items):
        paths.append(make_overlay( i , len(items)))
            
    return hv.NdOverlay({i:path for i,path in zip(items, paths)} , kdims=['TYPE'])

hv.DynamicMap(callback, streams=[selected])

# This shuffles the color cycle and works as the length NUM is unchanged
shuffled = list(range(NUM)) 
random.shuffle(shuffled)
selected.event(items=shuffled)

# Shrinking results in traceback, increasing gives wrong results
selected.event(items=list(range(NUM-1)))  

Traceback for NUM-1

--------------------------------------------------------------------------- AttributeError Traceback (most recent call last) in ----> 1 selected.event(items=list(range(NUM-1))) # Shrinking results in traceback, increasing gives wrong results

~/Desktop/development/holoviews/holoviews/streams.py in event(self, **kwargs)
387 """
388 self.update(**kwargs)
--> 389 self.trigger([self])
390
391 def update(self, **kwargs):

~/Desktop/development/holoviews/holoviews/streams.py in trigger(cls, streams)
156 with triggering_streams(streams):
157 for subscriber in subscribers:
--> 158 subscriber(**dict(union))
159
160 for stream in streams:

~/Desktop/development/holoviews/holoviews/plotting/plot.py in refresh(self, **kwargs)
613 stream_key = util.wrap_tuple_streams(key, self.dimensions, self.streams)
614
--> 615 self._trigger_refresh(stream_key)
616 if self.comm is not None and self.top_level:
617 self.push()

~/Desktop/development/holoviews/holoviews/plotting/plot.py in _trigger_refresh(self, key)
622 # Update if not top-level, batched or an ElementPlot
623 if not self.top_level or isinstance(self, GenericElementPlot):
--> 624 self.update(key)
625
626

~/Desktop/development/holoviews/holoviews/plotting/plot.py in update(self, key)
594 if len(self) == 1 and ((key == 0) or (key == self.keys[0])) and not self.drawn:
595 return self.initialize_plot()
--> 596 item = self.getitem(key)
597 self.traverse(lambda x: setattr(x, '_updated', True))
598 return item

~/Desktop/development/holoviews/holoviews/plotting/plot.py in getitem(self, frame)
259 if not isinstance(frame, tuple):
260 frame = self.keys[frame]
--> 261 self.update_frame(frame)
262 return self.state
263

~/Desktop/development/holoviews/holoviews/plotting/bokeh/element.py in update_frame(self, key, ranges, element)
2146 not subplot in self.dynamic_subplots):
2147 continue
-> 2148 subplot.update_frame(key, ranges, element=el)
2149
2150 if not self.batched and isinstance(self.hmap, DynamicMap) and items:

~/Desktop/development/holoviews/holoviews/plotting/bokeh/element.py in update_frame(self, key, ranges, element)
2112
2113 # Update plot options
-> 2114 plot_opts = self.lookup_options(element, 'plot').options
2115 inherited = self._traverse_options(element, 'plot',
2116 self._propagate_options,

~/Desktop/development/holoviews/holoviews/plotting/plot.py in lookup_options(cls, obj, group)
104 style_opts = None
105
--> 106 node = Store.lookup_options(cls.backend, obj, group)
107 if group == 'style' and style_opts is not None:
108 return node.filtered(style_opts)

~/Desktop/development/holoviews/holoviews/core/options.py in lookup_options(cls, backend, obj, group, defaults)
1226 def lookup_options(cls, backend, obj, group, defaults=True):
1227 # Current custom_options dict may not have entry for obj.id
-> 1228 if obj.id in cls._custom_options[backend]:
1229 return cls._custom_options[backend][obj.id].closest(obj, group, defaults)
1230 elif defaults:

AttributeError: 'NoneType' object has no attribute 'id'

The error occurs somewhere in the options system even though no options are explicitly used (of course there are the default options in the background). If there is a fundamental problem supporting NdOverlays that change how many items they contain, then a better traceback would help here.

[jbednar]

Does it have to be a stream, or would changing the number of objects manually also do it?

[philippjfr]

Not to say this doesn't have to be fixed but have you tried setting legend_limit=0 to force batching to be enabled?

[jlstevens]

I've had a go setting legend_limit=0 on the Overlay and NdOverlay with no change in the behavior...

[philippjfr]

I hadn't quite read your example, I'd say the organization is a bit odd, putting Overlays inside NdOverlays is rarely a great thing to do, especially when you're dealing with Paths which are much more efficiently drawn when combined. That said this turns out to be a relatively easy fix.

[jlstevens]

I'd say the organization is a bit odd, putting Overlays inside NdOverlays is rarely a great thing to do, especially when you're dealing with Paths which are much more efficiently drawn when combined.

I agree this simplified example is odd: it only aims to illustrate the issue I am encountering in a much more complex example where the structure makes more sense (with more element types involved in the overlay).

Anyway, glad to know this isn't a difficult fix!