I want to be able to visualize streaming data, e.g. from the Streaming Analytics service

[natashadsilva]

Expected behavior

I have an incoming stream of data, such as a stream of patient vital signs as described in this notebook .

I'd like to use PixieDust to display this data such that the graph is continuously updated :

Actual behavior

I don't see how this is possible using PixieDust at the moment, it seems to only support displaying a static Spark or Pandas DataFrames.

Steps to reproduce the behavior

[DTAIEB]

@natashadsilva do you know who we can easily simulate this streaming data? this would be helpful to build the corresponding pixiedust plugin

[natashadsilva]

@DTAIEB I created this DSX notebook for that purpose.
https://github.com/natashadsilva/samples/blob/master/Visualize%20data%20from%20Streams%20in%20PixieDust.ipynb
It demonstrates my use case.
You just need to create an instance of the streaming analytics service on Bluemix, and run the cells.

[natashadsilva]

@DTAIEB any outlook or comments for this feature?

[DTAIEB]

@natashadsilva hasn't made it to the top of the pile yet mostly due to resources, but it's getting there. Rough outlook would be mid July

[jinxuan]

So will it run on spark structure streaming right? Is mid-July scheduled as an expected time to release this version?

[DTAIEB]

@jinxuan This item is covering low level interfaces, defining how PixieDust will interact with Streaming data. The first commit (targeted at PixieDust 1.0.11) will include a few implementations: Kafka, IBM Streams and regular rest end point. Spark structure streaming will come later, however, you should be able to create your own PixieDust Streaming Adapter right away.