Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing real time streaming example #1775

Closed
Nemecsek opened this issue Aug 2, 2017 · 19 comments
Closed

Missing real time streaming example #1775

Nemecsek opened this issue Aug 2, 2017 · 19 comments
Labels
type: docs Related to the documentation and examples
Milestone

Comments

@Nemecsek
Copy link

Nemecsek commented Aug 2, 2017

Holoviews is great but I miss a real time streaming example.

It could probably be interesting for a lot of people: try to imagine all the Arduino/IoT aficionados that could show graphs with very little programming expertise.

This demo would show:

  1. how to embed a couple of Holoviews/Bokeh plots in a minimal flask web server (so they can manage multiple sources); I would use <iframe> to avoid useless complexities.

  2. how these plots would be updated automagically every X seconds.

Thanks

@philippjfr philippjfr added the type: docs Related to the documentation and examples label Aug 2, 2017
@jlstevens
Copy link
Contributor

We would love to add some streaming examples!

I think the flask suggestion is a good idea but the biggest problem I have is thinking of a source of live data that 1. everyone can easily understand 2. that everyone can access without needing 3rd party libraries (ideally) 3. that is robust but also free - ideally not some internet stream by some unreliable source that may vanish (or simply change) overnight.

If you have any suggestions, I would love to hear them!

@jordansamuels
Copy link
Contributor

Could we just use something generated by the local machine, e.g. an operating system performance metric?

@jlstevens
Copy link
Contributor

I had considered this before...e.g core temperatures, CPU load etc.

Unfortunately, I don't know of any reliable way to get this sort of information across machines/operating systems in a way that won't rely on some sort of 3rd party libraries (that are unlikely to be installed).

Maybe it would be easier to get some sound information from a microphone? I wonder if pyglet lets you do this, especially as that can be packaged as a single file...

@jbednar
Copy link
Member

jbednar commented Aug 2, 2017

The Bokeh spectrogram example might be a good starting point.

@jlstevens
Copy link
Contributor

@jbednar That is what I had in mind.

Unfortunately, pyaudio seems to be the standard way to do this (and what bokeh uses) but I am seeing various stackoverflow issues suggesting that the installation isn't trivial, at least on osx (pyaudio consists of bindings to some underlying library that also needs to be available).

@jlstevens
Copy link
Contributor

psutil looks like another reasonable option. My only worry is it seems to need a C compiler of some sort on Unix systems.

@jlstevens
Copy link
Contributor

I suppose we should replicate the bokeh spectogram example in HoloViews regardless of any issues getting pyaudio running...

@ea42gh
Copy link
Contributor

ea42gh commented Aug 2, 2017

an example of such a capability need not rely on a 'real data source': just generate some data that drive the display. The point is 'here is how to do this', not 'here is a neat example, and here is how to do this'?

@jlstevens
Copy link
Contributor

That is a good point - there isn't any difference between ongoing simulations and real streaming measurements from the perspective of HoloViews.

For example, the current boids example can be run indefinitely as an example of live data. It isn't streaming in the sense of a growing curve though and it obviously doesn't demonstrate the flask suggestion either.

@jbednar
Copy link
Member

jbednar commented Aug 2, 2017

The only problem with a fake data source is that it requires a chunk of code that can be distracting from what's really going on; we don't really want people to fake a data source. I'd favor having something pyaudio or psutils based that runs well when it runs, and not worry too much about whether it will run on every system; it's still a useful example anyway. Separately having a fake data source is also useful, for one that will always run but will be more confusing.

@jlstevens
Copy link
Contributor

Agreed. We don't need to chose one over the other - the more examples we have the better!

@jlstevens
Copy link
Contributor

Another idea would be to show network traffic. Not sure if there is a suitable cross-platform library for that though...

@stonebig
Copy link
Contributor

stonebig commented Oct 27, 2017

strange thing: replaying the example by downloading "Working_with_Streaming_Data.ipynb":

  • the existing bokeh plots in the notebook don't get cleared per a "restart and clear output", and are moving,
  • new plots area are created and run as expected, on the top of undeletable previous plots,
  • everything is normal if i redo the stream example on an empty notebook.

any idea what can cause this ? (first time I can't get an ouput cleared)
notebook_bokeh_holoview

@stonebig
Copy link
Contributor

stonebig commented Oct 28, 2017

ah, there are some "http://assets.holoviews.org/gifs/guides/user_guide/Streaming_Data/streamz1.gif" which are not cleared by the notebook "clear" option

@philippjfr
Copy link
Member

That's right, I've embedded gifs to demonstrate the streaming data in a static notebook. I have not added those to the notebook in the repo yet, but I'm going to have to do something. The problem is that they are indeed very annoying when running the notebook yourself.

@jlstevens
Copy link
Contributor

@philippjfr Maybe there is a way we can embed the gifs but only for doc building? Maybe we can specify the gif somehow e.g commented out HTML in the markdown cell which only gets uncommented for the docs?

@philippjfr
Copy link
Member

commented out HTML in the markdown cell

I've never worked out how to comment out HTML in notebook markdown and that would involve further buildsteps.

For now I may just go for something like this which will hide the gifs when you run it:

from IPython.display import display_javascript
display_javascript("$('img.gif').hide()") # Hide gifs

@jlstevens
Copy link
Contributor

All the proposed solutions here are pretty horrible, I agree!

@philippjfr philippjfr added this to the v1.10 milestone Jan 4, 2018
@philippjfr
Copy link
Member

I think this has been sufficiently addressed with a user guide about streaming data along. The issue about embedding gifs should be addressed separately.

@philippjfr philippjfr modified the milestones: v1.10, 1.9.3 Feb 10, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: docs Related to the documentation and examples
Projects
None yet
Development

No branches or pull requests

7 participants