GUI for drawing and monitoring NoFlo graphs #1

Closed
bergie opened this Issue Jul 15, 2011 · 35 comments

Projects

None yet

10 participants

@bergie
Member
bergie commented Jul 15, 2011

NoFlo needs a GUI that can be used for drawing and monitoring graphs. The idea there is that once NoFlo boots up a graph, it is "living software" that you can change and see how it is doing.

For example, the paths most commonly taken between nodes could be visualized with thicker lines, and processes that are down (database connection has gone away, a web service is down, whatever) could show their error state.

@bergie
Member
bergie commented Jul 15, 2011

We need a nice web graph library for doing this. Some options I've found:

@bergie
Member
bergie commented Jul 18, 2011

In essence a running FBP graph should be possible to visualize a bit like nuclear plant control rooms where you see the flow and throughput of processes with a simple glance.

@nka11
nka11 commented Jul 18, 2011

Had a look on all of the options.

My preference go to jsPlumb, cause it's the only one who provides svg visualization, and therefore, may be the most appropriate for dynamic process monitoring vizualisation.

additionally, it supports jQuery which is good for it.

So, +1 for jsPlumb

Regards

@bergie
Member
bergie commented Jul 18, 2011

@nka11 agreed, jsPlumb seems like the best option. It doesn't impose any particular editing UI, so we can implement our own UX. And it is MIT-licensed which fits NoFlo licensing.

WireIt would've been most feature-complete of the bunch, but I feel it tries to do too much.

@bergie
Member
bergie commented Aug 2, 2011

Current status of the GUI with jsPlumb:

Simple NoFlo network

We can now visualize existing networks, but editing them doesn't do anything yet.

To get here, do the following:

  • Start the NoFlo shell with $ noflo
  • Either load an existing graph (load file) or create a new one (new graphname)
  • Start HTTP server startserver 8003
  • Direct your browser to the server
@bergie
Member
bergie commented Oct 5, 2011

Met the developer of JointJS recently. The DEVS example is pretty good fit, including showing port names.

@mwawrusch

Hi everyone, new to NoFlo, here are my 2 cents:

After looking at all the libs mentioned I think it should be either JointJS or jsPlumb,

Requirements for me would definitely be MIT license and jquery compatibility.

jsPlumb seems to be more stable, but is hosted on google code, so contributing, fixing the diagramming layer is a pain. In addition to that it does not seem to be as feature rich as JointJS

The JointJS is on github which is a major plus, and it seems that it has a better architecture (I only glanced over it) and supports hierarchical components, which is hard to add after the fact. JointJS is using Raphael under the hood, which is a very solid product.

So to sum it up I would go with JointJS.

Regarding the UI: My experience having developed a workflow package and some diagramming packages in the far past is that simplicity is key. Nodes in, nodes out, may some color coding and an icon and one sentence description. More can be shown when a user hovers over a node (or does a single tap on the ipad)

Visual Debugging

People in general have a hard time visualizing flow once you have more than a couple nodes, which is why visual debugging (e.g. animating the flow) is nice for marketing but not often used in practice. If debugging is needed users are mostly in an aggravated state and just want to get the job done, which is why lists are preferred (events in, events out+ errors + filtering)

@bergie
Member
bergie commented Oct 11, 2011

@mwawrusch: lists are good for debugging, but some sort of visualization (throughput, number of packets processed) would probably work better for getting an overview of the state of a running application.

Agreed that JointJS is a closer match to the needs of FBP visualization, and I'm, also a bit allergic on Google Code.

I also heard an interesting idea of visualizing ports as jigsaw puzzle connectors. That way you could directly see what components fit together. But that is something to be explored later.

@jrz
jrz commented Nov 24, 2011

Just a question outside of this project.
What other pros/cons are there between jointjs and jsplumb?

The demos of jsplumb seem less nicely designed, but the interface is a lot better by default (hovers etc and performance)

@bergie
Member
bergie commented Nov 28, 2011

@jrz in nutshell, jsPlumb appears to be more mature, but JointJS has some features that we really would like to have in NoFlo (like named ports).

A big difference is that in jsPlumb nodes are regular DOM elements (DIVs, for instance), and only the edges are drawn with canvas. In JointJS the whole graph is a canvas. It appears the jsPlumb approach performs slightly better with large graphs.

Another potential differentiating point may be mobile support. I've been able to edit jsPlumb graphs using the HP TouchPad browser.

@jrz
jrz commented Nov 28, 2011

Hi bergie, thanks for the explanation.
I had already started with jsPlumb, and it indeed seems that jsPlumb is only for connecting elements.
While it works for what I'm doing, I did run into the hierarchical issue though.
I like that I can use DOM to do some styling, and use jquery draggables.

@mwawrusch

An update: jsPlumb is now on github, and I really like how it progressed over time. So I am changing my preference to it.

@bergie
Member
bergie commented Feb 27, 2012

There is also an effort to make DrawFBP able to create NoFlo networks. While we'll still also want a nice web-based editor, this is also a great thing for having a UI.

@mwawrusch

Thanks for the link, going to watch the video now. One more thing: You might want to create the organization noflo on github , even if you don't use it yet it might prove to be useful in the future.

@bergie
Member
bergie commented Feb 27, 2012

Interesting, this new demo of jsPlumb appears to support named ports.

@mwawrusch

Yep, they did a great job. I played with the lib for an hour or so and found only one bug so far, which is a good sign. Downside right now is no support for touch events on the IPad, but that could be added. I need something sooner than later, so I am probably going to try to whip something up using jsPlumb.

@bergie
Member
bergie commented Feb 27, 2012

@mwawrusch great! I'd be happy to integrate any results of that :-)

@mwawrusch

Damned, I got shanghaied into an OS project ;)

@bergie
Member
bergie commented Oct 22, 2012

It looks like http://meemoo.org/dataflow/ will be the way to go

@bergie
Member
bergie commented Nov 17, 2012

As required by #49, the UI work has been moved to a separate repository: https://github.com/bergie/noflo-ui

@mchogan
mchogan commented Jan 16, 2013

Have you made a decision on Canvas vs SVG? Where is the best place to discuss and ask questions that maybe aren't issues?

@bergie
Member
bergie commented Jan 22, 2013

@mchogan I'm starting to lean towards SVG. Works better on high-res displays, and probably also performs better in the long run.

@bergie
Member
bergie commented Feb 8, 2013

Here is my latest thinking on the matter: http://bergie.iki.fi/blog/inspiration-for-fbp-ui/

@mchogan
mchogan commented Feb 8, 2013

The subway map metaphor is really interesting.

I like a map metaphor in general, such as the interface of Google Maps that allows zooming in and out.

What I like about a map interface is that detail can be either shown or abstracted in meaningful ways. So I can show a node as a coordinate, a city, a country, a continent, a planet, but each is just a view-appropriate representation of a single node.

Another powerful thing about that sort of representation is that it aids refactoring of code. Maybe I don't know the details of the US, but I know I want to visit the Gand Canyon and Empire State Building. I can draw a node called US and give it those two attributes. Later, I can decide the huge US node is unwieldy and create sub-nodes inside of US for Arizona and New York and move the attributes to those sub-nodes, which are then easier to reuse across programs.

From my work with Canvas and SVG so far, I think it would be far more achievable to implement an "infinite map" or "zoom user interface" in HTML5 Canvas (for example with Kinetic.js) than with SVG.

Check out a simple Canvas site I made and bump up the number of paint dabs and it keeps running. Slowdown starts detracting from the animation up around 1000 nodes. I've not tried an equivalent exercise with SVG.

mchogan.com/html5/i-still-use-canvas/

On Feb 8, 2013, at 9:30 AM, Henri Bergius notifications@github.com wrote:

Here is my latest thinking on the matter: http://bergie.iki.fi/blog/inspiration-for-fbp-ui/


Reply to this email directly or view it on GitHub..

@bergie
Member
bergie commented Jun 19, 2013

screenshot 2013-06-19 at 11 28 48

Some interesting progress in https://github.com/forresto/dataflow-noflo/

@kenhkan
Member
kenhkan commented Jun 21, 2013

This looks really sweet! I like that it's pretty basic-feature-complete with the menu bar. One quick suggestion is to include the component's name in the box so it's clear what you're dealing with without opening the Source panel.

So this would obviously be a "graph" and we'd be able to add the ports to export? Or when you save it's automatically exporting unbound ports?

All in all, this is an amazing feat so far!

@bergie
Member
bergie commented Jun 21, 2013

@kenhkan for showing components there is forresto/dataflow-noflo#15

Port exporting is part of the requirements in forresto/dataflow-noflo#11

@sirwobin
Member

The dataflow-noflo demo looks quite impressive! That interface is quite familiar; relational data processing products that are very widely used use the same sort interface. Informatica and SQL Server Information Services come to mind.

@tomByrer
tomByrer commented Aug 8, 2013

TL;DR: Could link colors & connector types be considered to signify data types please?

I don't know how solidified your are on the GUI, but here is a "Flow-Programming" platform I've programmed on for 5 years called FlowStone:
examplegui_from_flowstone
The colors & symbols on the modules' links and connector points signify what kind of data they carry; blue is for 'streaming DSP' data (usually audio) that is Packed Single Floats (SSE), green is for triggered (on demand) data, the circular (F) green circles are for Floats, (I) Integers, odd-shaped {F} connectors mean is it is a Float-Array, etc.
In the picture above I selected the "Wet+Dry" module. Notice how there are extra controls that appear when the module is selected (I'm not suggesting you to add that). Let's look inside that module:
examplegui_from_flowstone_inside_wet dry
As you can see, FlowStone tends to be more of a GUI programming system than NoFlo is, though some direct coding is possible.

I'm not suggesting you need to rework everything, but I do find the extra colors & connector symbols to be helpful when quickly scanning a program to understand what data is being transferred between modules. There are also 'empty'/blank connectors so you know when a connector is not active.

Thanks for all your hard work; I'm a KickStarter supporter!

@arboleya
arboleya commented Aug 9, 2013

@tomByrer Looks great, I'd like to test it. Any kind of free license for OSS?

@tomByrer

Anyone can download a more or less fully functioning copy: http://www.dsprobotics.com/download.php
You can download the programs/'schematics' (*.fsm) from FlowStone's forum. The old forum & WIKI has many many *.osm schematics.

I used to be a super-user with SynthMaker (the old version), so let me know if you have any questions!

Edit:

platform I've programmed on for 5 years

I did not write the actual platform, just programed on it ;)

@bergie
Member
bergie commented Sep 25, 2013

Closing this issue here as we now can track the progress in https://github.com/noflo/noflo-ui and https://github.com/noflo/noflo-ui-server

@bergie bergie closed this Sep 25, 2013
@bergie
Member
bergie commented Oct 4, 2013

Update on the state of things: http://bergie.iki.fi/blog/noflo-update/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment