Support for visualization/analysis/debugging tools

[raimohanska]

In the feature/dependency-graph-dispatch, I've made some improvement that will enable the implementation of development tools that can, for instance, visualize the event network and event flow going on. The improvements include

observable.deps()

Returns an array of dependencies that the Observable has. For instance, for a.map(function() {}), the deps would return [a]. This method returns the "visible" dependencies only, skipping internal details. This method is thus suitable for visualization tools.

Internally, many combinator functions depend on other combinators to create intermediate Observables that the result will actually depend on. The deps method will skip these internal dependencies.

observable.toString()

Returns a nice textual presentation of the stream. Covered in #265.

observable.internalDeps()

Returns the true dependencies of the observable, including the intermediate "hidden" Observables. This method is for Bacon.js internal purposes but could be useful for debugging/analysis tools as well.

observable.desc()

Returns a structured version of what toString returns. The structured description is an object that contains the fields context, method and args. For example, for Bacon.fromArray([1,2,3]).desc() you'd get

{ context: Bacon, method: "fromArray", args: [[1,2,3]] }

Bacon.spy(f)

Adds your funcion as a "spy" that will get notified on all new Observables. This will allow a visualization/analytis tool to spy on all Bacon activity. Current implementation just calls your function, but maybe, it should allow you to wrap all created Observables in your own wrapper to make spying even easier? This would enable AOFRP (aspect oriented functional reactive programming) lol.

I'm planning to include this improvement in release 0.7.0

[raimohanska]

We should really look into allowing the wrapping of all created observables in Bacon.spy. That would allow tools to spy on all observables without forcing subscription. The alternative to that would be "subscribeWeakly" or something similar.

[raimohanska]

After some thought, I'm starting to think that wrapping is not such a good idea:

1. It will create an extra layer on top of all Observables, and thus affect the very depedency graph the Tool might be willing to visualize
2. It doesn't work so well for non-standard Observables like Bacon.Bus or Bacon.Model (external lib)

So, maybe we should try the Weak Subscription thing as in #251 ?

[rassie]

This is nice so far, however there is currently no way to name streams which would make analyzing dependency trees so much easier. Any plans to allow that somehow?

[rassie]

Oh, that was easy. Just doing something like stream._name = "name" at definition time has been enough.

[rassie]

Still, in cases where a stream is not assigned to a variable, it seems like a shortcut like stream.name("name") which returns the stream just like .log would be very nice.

[rassie]

On a connected note, it seems .deps() and .internalDeps() is always the same, e.g. an additional .when stream to each .combineAsArray stream is being returned in both methods. Not sure if it's a bug, will investigate more.

[raimohanska]

I'm quite close to exclude this feature from release 0.7.0, because it's not quite ready yet. What we need is a way to observe the events and lifecycle of all Observables without forcing subscriptions on them (i.e. messing up their lifecycle).

If we fully implemented #276 and #251, I think we would be ready. But we may release 0.7.0 without Bacon.spy() to get #107 and #272 out of the oven. Opinions?

[raimohanska]

Naah, I'm not gonna wait for this. Will release 0.7.0 and get back to this one afterwards.

[rassie]

I certainly can live without this for now. However, optional graph building (I'm using dagre-d3) is a very nice debugging tool, especially for visual checking if something is wrong in stream construction logic, so I'd be happy to see this finished in the future.

[rassie]

As mentioned in #449: with this functionality it's already possible to build a somewhat generic introspection/debugging visualization tool. What we need is finalization of this feature and probably something like #251 to make it completely invisible.

I've so far experimented with directed graphs (e.g. with dagre-d3 or vis.js) which come out very nice except for dependencies/internalDeps (probably a separate conversation). I've also adapted your visualization technique from Bacon.Game for a presentation I've given recently, which worked nicely too.

What I haven't done yet, but would like to, is something to visualize the actual dataflow, i.e. highlight current values in the directed graph. Another wish in that direction is to re-do https://github.com/baconjs/bacon.js/wiki/Diagrams with these visualization techniques and introduce interactive examples for as many combinators as possible.

[raimohanska]

As mentioned in #454, I made an attempt at a new spy system in aec6862

The idea is that a spy object contains methods newObservable and newEvent that will be called on creation of a new observable and dispatching of an event. This should allow the spy to register all Observables and Events without affecting their behaviour. Is this any good? Be honest.

[rassie]

I gotta test it out on my production code, which I can do tomorrow. However, a first glance suggests it's good. On an another test base at home, I've ported my directed graph visualization to the new spys and they work at least as good as the old ones. Getting a notification for every event is great and I hope it'll be a great help for visualizing event propagation.

[rassie]

A couple of notes from production:

• Crash when newEvent is not defined (probably the same when newObservable not defined)
• Same error as in Errors in UpdateBarrier with a generic error handler #454.

[raimohanska]

Thanks, @rassie! This is of course a breaking change in the sense that your old spies won't work; you need to have {newObservable, newEvent}. Regarding #454, nothing has changed: if you subscribe in your spy, you'll get errors. On the other hand, now there shouldn't be a need to subscribe, as you'll get all the events through your newEvent handler anyway.

[rassie]

Hmm, you are right, I've missed that part about not needing to subscribe. With if (value.isError()) no errors occur, however, I haven't had an Error event yet ;)