GUI for drawing and monitoring NoFlo graphs

[bergie]

NoFlo needs a GUI that can be used for drawing and monitoring graphs. The idea there is that once NoFlo boots up a graph, it is "living software" that you can change and see how it is doing.

For example, the paths most commonly taken between nodes could be visualized with thicker lines, and processes that are down (database connection has gone away, a web service is down, whatever) could show their error state.

[bergie]

We need a nice web graph library for doing this. Some options I've found:

• jsPlumb (see example), MIT license, works with jQuery, MooTools or YUI
• Draw2D (see example), LGPL but interestingly website says purchase. Works with MooTools
• UMLCanvas (see example)
• WireIt (see example), MIT license, YUI

[bergie]

Here is also an interesting example from the proprietary world

[bergie]

In essence a running FBP graph should be possible to visualize a bit like nuclear plant control rooms where you see the flow and throughput of processes with a simple glance.

[nka11]

Had a look on all of the options.

My preference go to jsPlumb, cause it's the only one who provides svg visualization, and therefore, may be the most appropriate for dynamic process monitoring vizualisation.

additionally, it supports jQuery which is good for it.

So, +1 for jsPlumb

Regards

[bergie]

@nka11 agreed, jsPlumb seems like the best option. It doesn't impose any particular editing UI, so we can implement our own UX. And it is MIT-licensed which fits NoFlo licensing.

WireIt would've been most feature-complete of the bunch, but I feel it tries to do too much.

[bergie]

Current status of the GUI with jsPlumb:

We can now visualize existing networks, but editing them doesn't do anything yet.

To get here, do the following:

• Start the NoFlo shell with $ noflo
• Either load an existing graph (load file) or create a new one (new graphname)
• Start HTTP server startserver 8003
• Direct your browser to the server

[bergie]

Met the developer of JointJS recently. The DEVS example is pretty good fit, including showing port names.

[mwawrusch]

Hi everyone, new to NoFlo, here are my 2 cents:

After looking at all the libs mentioned I think it should be either JointJS or jsPlumb,

Requirements for me would definitely be MIT license and jquery compatibility.

jsPlumb seems to be more stable, but is hosted on google code, so contributing, fixing the diagramming layer is a pain. In addition to that it does not seem to be as feature rich as JointJS

The JointJS is on github which is a major plus, and it seems that it has a better architecture (I only glanced over it) and supports hierarchical components, which is hard to add after the fact. JointJS is using Raphael under the hood, which is a very solid product.

So to sum it up I would go with JointJS.

Regarding the UI: My experience having developed a workflow package and some diagramming packages in the far past is that simplicity is key. Nodes in, nodes out, may some color coding and an icon and one sentence description. More can be shown when a user hovers over a node (or does a single tap on the ipad)

Visual Debugging

People in general have a hard time visualizing flow once you have more than a couple nodes, which is why visual debugging (e.g. animating the flow) is nice for marketing but not often used in practice. If debugging is needed users are mostly in an aggravated state and just want to get the job done, which is why lists are preferred (events in, events out+ errors + filtering)

[bergie]

@mwawrusch: lists are good for debugging, but some sort of visualization (throughput, number of packets processed) would probably work better for getting an overview of the state of a running application.

Agreed that JointJS is a closer match to the needs of FBP visualization, and I'm, also a bit allergic on Google Code.

I also heard an interesting idea of visualizing ports as jigsaw puzzle connectors. That way you could directly see what components fit together. But that is something to be explored later.

[jrz]

Just a question outside of this project.
What other pros/cons are there between jointjs and jsplumb?

The demos of jsplumb seem less nicely designed, but the interface is a lot better by default (hovers etc and performance)

[bergie]

@jrz in nutshell, jsPlumb appears to be more mature, but JointJS has some features that we really would like to have in NoFlo (like named ports).

A big difference is that in jsPlumb nodes are regular DOM elements (DIVs, for instance), and only the edges are drawn with canvas. In JointJS the whole graph is a canvas. It appears the jsPlumb approach performs slightly better with large graphs.

Another potential differentiating point may be mobile support. I've been able to edit jsPlumb graphs using the HP TouchPad browser.

[jrz]

Hi bergie, thanks for the explanation.
I had already started with jsPlumb, and it indeed seems that jsPlumb is only for connecting elements.
While it works for what I'm doing, I did run into the hierarchical issue though.
I like that I can use DOM to do some styling, and use jquery draggables.

[mwawrusch]

An update: jsPlumb is now on github, and I really like how it progressed over time. So I am changing my preference to it.

[bergie]

There is also an effort to make DrawFBP able to create NoFlo networks. While we'll still also want a nice web-based editor, this is also a great thing for having a UI.

[mwawrusch]

Thanks for the link, going to watch the video now. One more thing: You might want to create the organization noflo on github , even if you don't use it yet it might prove to be useful in the future.

[bergie]

Interesting, this new demo of jsPlumb appears to support named ports.

[mwawrusch]

Yep, they did a great job. I played with the lib for an hour or so and found only one bug so far, which is a good sign. Downside right now is no support for touch events on the IPad, but that could be added. I need something sooner than later, so I am probably going to try to whip something up using jsPlumb.

[bergie]

@mwawrusch great! I'd be happy to integrate any results of that :-)

[mwawrusch]

Damned, I got shanghaied into an OS project ;)

[bergie]

It looks like http://meemoo.org/dataflow/ will be the way to go

[bergie]

As required by #49, the UI work has been moved to a separate repository: https://github.com/bergie/noflo-ui

[mchogan]

Have you made a decision on Canvas vs SVG? Where is the best place to discuss and ask questions that maybe aren't issues?

[bergie]

@mchogan I'm starting to lean towards SVG. Works better on high-res displays, and probably also performs better in the long run.

[bergie]

Here is my latest thinking on the matter: http://bergie.iki.fi/blog/inspiration-for-fbp-ui/

[mchogan]

The subway map metaphor is really interesting.

I like a map metaphor in general, such as the interface of Google Maps that allows zooming in and out.

What I like about a map interface is that detail can be either shown or abstracted in meaningful ways. So I can show a node as a coordinate, a city, a country, a continent, a planet, but each is just a view-appropriate representation of a single node.

Another powerful thing about that sort of representation is that it aids refactoring of code. Maybe I don't know the details of the US, but I know I want to visit the Gand Canyon and Empire State Building. I can draw a node called US and give it those two attributes. Later, I can decide the huge US node is unwieldy and create sub-nodes inside of US for Arizona and New York and move the attributes to those sub-nodes, which are then easier to reuse across programs.

From my work with Canvas and SVG so far, I think it would be far more achievable to implement an "infinite map" or "zoom user interface" in HTML5 Canvas (for example with Kinetic.js) than with SVG.

Check out a simple Canvas site I made and bump up the number of paint dabs and it keeps running. Slowdown starts detracting from the animation up around 1000 nodes. I've not tried an equivalent exercise with SVG.

mchogan.com/html5/i-still-use-canvas/

On Feb 8, 2013, at 9:30 AM, Henri Bergius notifications@github.com wrote:

Here is my latest thinking on the matter: http://bergie.iki.fi/blog/inspiration-for-fbp-ui/

—
Reply to this email directly or view it on GitHub..

[bergie]

Some interesting progress in https://github.com/forresto/dataflow-noflo/

[kenhkan]

This looks really sweet! I like that it's pretty basic-feature-complete with the menu bar. One quick suggestion is to include the component's name in the box so it's clear what you're dealing with without opening the Source panel.

So this would obviously be a "graph" and we'd be able to add the ports to export? Or when you save it's automatically exporting unbound ports?

All in all, this is an amazing feat so far!

[bergie]

@kenhkan for showing components there is forresto/dataflow-noflo#15

Port exporting is part of the requirements in forresto/dataflow-noflo#11

[sirwobin]

The dataflow-noflo demo looks quite impressive! That interface is quite familiar; relational data processing products that are very widely used use the same sort interface. Informatica and SQL Server Information Services come to mind.

[tomByrer]

TL;DR: Could link colors & connector types be considered to signify data types please?

I don't know how solidified your are on the GUI, but here is a "Flow-Programming" platform I've programmed on for 5 years called FlowStone:

The colors & symbols on the modules' links and connector points signify what kind of data they carry; blue is for 'streaming DSP' data (usually audio) that is Packed Single Floats (SSE), green is for triggered (on demand) data, the circular (F) green circles are for Floats, (I) Integers, odd-shaped {F} connectors mean is it is a Float-Array, etc.
In the picture above I selected the "Wet+Dry" module. Notice how there are extra controls that appear when the module is selected (I'm not suggesting you to add that). Let's look inside that module:

As you can see, FlowStone tends to be more of a GUI programming system than NoFlo is, though some direct coding is possible.

I'm not suggesting you need to rework everything, but I do find the extra colors & connector symbols to be helpful when quickly scanning a program to understand what data is being transferred between modules. There are also 'empty'/blank connectors so you know when a connector is not active.

Thanks for all your hard work; I'm a KickStarter supporter!

[arboleya]

@tomByrer Looks great, I'd like to test it. Any kind of free license for OSS?

[tomByrer]

Anyone can download a more or less fully functioning copy: http://www.dsprobotics.com/download.php
You can download the programs/'schematics' (*.fsm) from FlowStone's forum. The old forum & WIKI has many many *.osm schematics.

I used to be a super-user with SynthMaker (the old version), so let me know if you have any questions!

Edit:

platform I've programmed on for 5 years

I did not write the actual platform, just programed on it ;)

[jonnor]

A kickstarter is in progress now: http://www.kickstarter.com/projects/noflo/noflo-development-environment

[bergie]

Closing this issue here as we now can track the progress in https://github.com/noflo/noflo-ui and https://github.com/noflo/noflo-ui-server

[bergie]

Update on the state of things: http://bergie.iki.fi/blog/noflo-update/