New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduction of Streams API #832

Merged
merged 35 commits into from Aug 31, 2016

Conversation

Projects
None yet
3 participants
@jlstevens
Member

jlstevens commented Aug 25, 2016

This is the first PR of many that will introduce an exciting new feature to HoloViews 1.7, namely interactive streams. Everything is very much work-in-progress at this stage.

Streams will make it easy to enable interactive visualizations with HoloViews by making it possible for HoloViews plots to react to events. These events may be generated on the server-side from Python or on the client-side via JavaScript.

In this PR, the focus is on the former (no JavaScript) and future PRs will enable interaction in the browser (using the Bokeh backend). Note that the design of streams ensures they are backend agnostic as possible (like everything else in HoloViews!).

The key points about streams:

  • Streams are simply Parameterized objects whose parameters change over time in response to events.
  • These events involve specifying new parameter values using the update method.
  • A list of stream instances can be passed to DynamicMap to enable interactivity. The stream parameters are passed as keyword arguments to the DynamicMap callable when update events occur.
  • Streams have their own callbacks (e.g user defined Python functions) that are also invoked with the stream parameter values.

This PR is currently work-in-progress but here is a quick preview. Note that this example will work with the matplotlib backend after a future PR is merged to update the plotting system:

image

Right now, a dummy kdim ('Ignored') is declared which won't be needed in future.

Next the following setup is currently needed. In future, this will happen automatically and will be hidden from the user:

image

Now when you call update on the stream:

m1.update(dict(y=-0.5, x=-0.5))

The plot responds to this event:

image

We plan to include the following streams:
AxisRangeX, AxisRangeY, AxisRangeXY, BoxSelection as well as more specialized streams such as ElementSelector and DataSelector. We'll also make it as easy as possible for users to define their own custom stream implementations.

Currently some of our plans are outlined on the wiki here and here. These pages are likely to go stale quickly as the implementation evolves.

Exciting stuff and obviously a lot more to come!

@jlstevens jlstevens added this to the v1.7.0 milestone Aug 25, 2016

@jlstevens

This comment has been minimized.

Member

jlstevens commented Aug 31, 2016

I think this PR is ready for review. Of course, this is only a first cut at streams and more work is needed to integrate them with the plotting system etc. What this PR should do is make streams available so that integration work can begin.

param.constant = False
self.set_param(**kwargs)
for param in self.params().values():
param.constant = True

This comment has been minimized.

@philippjfr

philippjfr Aug 31, 2016

Member

So parameters can't be set directly, ever? How would this play with paramnb, where the widgets set the parameter values directly. I was hoping I could simply attach Stream.update as a callback on the paramnb widgets and have it just work, but if the params are constant it will never work.

This comment has been minimized.

@jlstevens

jlstevens Aug 31, 2016

Member

I think that is fine: just subclass from Stream without declaring the parameters as constant to create a parameterized object to use with paramnb.

What I think I can fix is to restore the constant state to what it was originally. That way, declaring stream parameters them as constant is recommended but not mandatory and they suddenly become constant when update is used.

This comment has been minimized.

@philippjfr

philippjfr Aug 31, 2016

Member

Sounds reasonable.

This comment has been minimized.

@jbednar

jbednar Aug 31, 2016

Member

I'm a bit alarmed by the suggestion to subclass from Stream for creating a parameterized object for use with paramnb -- the whole point of paramnb is to be able to display Jupyter-editable parameter widgets for parameters that are defined independently of any GUI or plotting system. The underlying parameterized object sometimes is created just for the GUI, but in many cases is something that doesn't know or need to know anything about anything to do with GUIs or plotting. Maybe in this particular case it is fine, because people using paramnb together with HoloViews streams are already doing things tied to plotting, but it does alarm me!

This comment has been minimized.

@philippjfr

philippjfr Aug 31, 2016

Member

It's just one way of doing it, you can easily create a callback to attach to the paramnb widgets which copies the parameterized objects parameters over to a stream. All I wanted to ensure here is that you can create a parameterized object that works both as a Stream and as the input to paramnb.widgets.

@@ -454,6 +463,12 @@ class DynamicMap(HoloMap):
""")
def __init__(self, callback, initial_items=None, **params):
# Set source to self if not already specified
for stream in params.get('streams',[]):

This comment has been minimized.

@philippjfr

philippjfr Aug 31, 2016

Member

This should happen after the parameters have been instantiated. I wouldn't recommend it but someone could set a number of default streams directly on DynamicMap.

This comment has been minimized.

@jlstevens

jlstevens Aug 31, 2016

Member

Agreed. I'll fix this now.

"""
# Union of stream values
items = [stream.value.items() for stream in streams]
union = dict(kv for kvs in items for kv in kvs)

This comment has been minimized.

@philippjfr

philippjfr Aug 31, 2016

Member

What happens when there are clashes? Shouldn't it warn at least?

This comment has been minimized.

@jlstevens

jlstevens Aug 31, 2016

Member

Sure. I was planning to do that and then forgot.

Once issue is that self.warning seems to be showing up in the console and not the notebook...

This comment has been minimized.

@philippjfr

philippjfr Aug 31, 2016

Member

Somehow it works for Jim, not sure why.

This comment has been minimized.

@jbednar

jbednar Aug 31, 2016

Member

It doesn't always work; sometimes I see messages from param on the console. But usually they go to the notebook.

retval = self.callback(*args)
# Additional validation needed to ensure kwargs don't clash
kwarg_items = [s.value.items() for s in self.streams]

This comment has been minimized.

@philippjfr

philippjfr Aug 31, 2016

Member

Same as below, no warning for clashing kwargs?

@philippjfr

This comment has been minimized.

Member

philippjfr commented Aug 31, 2016

Apart from my three comments it looks good. Doesn't need to be finalized right now anyway. Once we've hooked everything up we can still reconsider some of the API.

@jlstevens

This comment has been minimized.

Member

jlstevens commented Aug 31, 2016

I believe I've address your three comments. Ready for final review/merge.

# Currently building a simple set of subscribers
groups = [stream.subscribers + stream._hidden_subscribers for stream in streams]
subscribers = set(s for subscribers in groups for s in subscribers)
for subscriber in subscribers:

This comment has been minimized.

@philippjfr

philippjfr Aug 31, 2016

Member

Hmm, shouldn't some order be maintained, so that the subscribers defined on an individual stream are at least executed in sequence and the _hidden_subscribers are executed after all others?

Something like:

from .core import util

groups = [sub for stream in streams for sub in stream.subscribers]
hidden = [sub for stream in streams for sub in stream._hidden_subscribers]
for subscriber in util.unique_iterator(groups+hidden):
    ...

This comment has been minimized.

@jlstevens

jlstevens Aug 31, 2016

Member

Yes, that makes sense mainly because we know that _hidden_subscribers is the sort of thing we want to batch and execute last.

This comment has been minimized.

@philippjfr

philippjfr Aug 31, 2016

Member

Right, currently it is not well defined when the plot redraw would happen.

@philippjfr

This comment has been minimized.

Member

philippjfr commented Aug 31, 2016

Looks good, tests will pass. Now we just need to hook it all up, exciting!

@philippjfr philippjfr merged commit 6c185e8 into master Aug 31, 2016

2 of 4 checks passed

continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
continuous-integration/travis-ci/push The Travis CI build is in progress
Details
coverage/coveralls Coverage increased (+22.6%) to 69.11%
Details
s3-reference-data-cache Test data is cached.
Details

@philippjfr philippjfr removed the in progress label Aug 31, 2016

@jbednar jbednar deleted the streams_API branch Aug 31, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment