Integrating new UI

[baransu]

I'm doing research on adding erlangpl-ui as a new frontend. Only thing I need from backend is serving static folder at localhost:8000 as it is right now.
I'll add erlangpl-ui as subrepo which will be fetched during build process and included in place of current UI.

Only thing I have problem understanding is how erl_3d is included in current frontend because plugin provides frontend as well. With new version it would be hard or impossible, and we're thinking about replacing it with 2nd or 3rd level view of vizceral. But having 3D visualization is nice and I can work on new version of 3D renderer for that which will be available at /messages within new UI. In that case erl_3d can only serve as backend plugin and all client code can be placed in erlangpl-ui or new repo.

What have to be done within this and epl_3d repo to provide new UI?

/ cc @michalslaski @arkgil

[arkgil]

I'm glad you raised this issue, because I have questions regarding plugin system with new UI.

To answer your question (with a bit of simplification), whenever plugin is loaded (each plugin is OTP application), all the files from its priv/htdocs directory are served at the path matching plugin's name. For example, epl_3d has priv/htdocs/index.html file, when we start erlangpl, this file will be server at /epl_3d/index.html path.

epl_3d won't be used in its current form. Message passing visualisation, as you said, will be replaced by vizceral views. I have already implemented plugin which shows supervision trees of applications on monitored nodes (erlanglab/epl_sup_tree). I think Michal is thinking about using 3d view in some other way.

If I understand correctly, with the new UI, the only way to introduce new view, is to implement it directly in erlangpl-ui codebase, right?

[baransu]

Yes and no. We can compose UI from smaller plugins but it have to be on the client side where plugin is added.

I can wrap existing code into React and use it that way. It's similar how vizceral-react works. We can provide any plugin we'd like to add in the future with it's React or Elm component which can be easily integrated with erlangpl-ui, but I'm not sure if we should keep server/client code in one repo or create new repos for erlang code and for frontend use something like https://github.com/lerna/lerna

Only concern I have about it is which part of the application should connect to backend. I think it could be wise to allow every plugin to connect on it's own, but they should have ability to communicate with main part as well.

[arkgil]

I think that these are very valid concerns. I'd really love to figure out complete plugin system, such as anybody could just provide a path to their plugin in a filesystem, both frontend and backend would be loaded and work beautifully. However it would probably take us a lot of time 😄

I think that at this point (given the amount of time we have) a reasonable solution would be:

• as you said, separate frontend and backend code of each default plugin (which for now are: epl_dashboard, epl_sup_tree and vizceral-message-passing-thingy)
• add frontend code of those plugins as dependencies to erlangpl-ui
• add backend code of those plugins as dependencies to erlangpl
• and find a way to package it all up into one executable (which is done for 0.3)

Later on, when we have a concept of easy to use plugin system, we can split things up. But for now I'd be for making it work. What do you thing?

Could you expand your last point a bit? I am not sure I get what you mean.

[baransu]

Right now erlangpl-ui has one state which contains all data. Every component can opt-in for state change and relevant view update will happen. This solution force us to provide new state "extension" with every new plugin with it's own WebSocket connection and state changes.

The only problem I can imagine is case when one plugin require data from other plugin. But yes, we can specify that this plugin require peer dependency and it will solve problem.
It's design problem, not implementation because when peer plugin is not present we can simply show info about that.

So right now I'll focus on making epl_dashboard as separate plugin. I'll try to use monorepo approach to erlangpl-ui so all default frontend code will be within this repo.

I have one question though. If vizceral-message-passing-thingy will be responsible for vizceral view, what view epl_sub_tree should have? Should we have 2 plugins based on vizceral? I was thinking about one view with transitions to 2nd and 3rd level, the same way as PoC worked.

[arkgil]

Vizceral view will show message passing between nodes/processes. epl_sup_tree is only for displaying supervision trees (similarly to what Erlang's observer does). So it is completely unrelated to Vizceral view.

I think that our current approach with subscriptions to state changes is fine. In addition I have added a epl:command/2 API, which can be used to call any function on remote node. There is one caveat I see - multiple WebSocket connections. It would be nice to route messages to plugins on the backend, the same way we can route them on the frontend using topic field in message payload.

[arkgil]

I'm currently experimenting with different library for rendering supervision trees, so I'll try to implement that in erlangpl-ui straight away.

[baransu]

What lib are you using?

[arkgil]

sigma.js looks promising, although I have problems with embedding it into simple div, not to mention React (I really can't do frontend 😄 ). I chose that lib because for me 2d rendering of supervision trees is more readable than 3d.

[baransu]

Are you experimenting on current erlangpl-ui or new project only to test this lib?

[arkgil]

First I used slightly modified epl_3d UI for epl_sup_tree, so trees were rendered in 3d. Then I tried to use sigma for rendering, but I couldn't make it display anything, so now I try with simple static pages and try to get to know their API. I haven't tried erlangpl-ui yet, although I think I've understood enough to create a simple component and embed it into current view.

[baransu]

Ok, if you have any problems let me know.

I'll focus on making erlangpl-ui pluggable then.

[arkgil]

@baransu do you have any ideas on working with multiple socket connections on the frontend?

[baransu]

I think we can easily create new WebSocket connection for every plugin route and it should just work. It even should work the same with multiple connections to the same route.

[baransu]

@arkgil How things going with vizceral-message-passing-thingy since this is key part of whole project.

PoC has code written by Jakub by Michal wasn't sure about merging it into project.
If you could focus on Erlang code for that I'll work more on vizceral frontend as well as epl-sub-tree's fontend since I have no idea about Erlang.

[michalslaski]

@baransu I'm now focusing on adding support for traffic view. First level should use information from Erlang's net_kernel to visualize traffic volumes between nodes. Second level should use information from Erlang tracer to visualize traffic volumes between processes. Third level should use information from the process_info/2 BIF to present state of the process.

[michalslaski]

@arkgil I did some research on 2D visualizations of graphs/trees and would like to suggest to try the JavaScript library by Andrei Kashcha https://github.com/anvaka/VivaGraphJS

I'm not sure if it fits your supervision tree view, but if you have issues with sigma.js, maybe it will turn out as an alternative.

[arkgil]

It looks really nice, similarly to sigma it offers a lot of customisation. I'll try it out, thanks!

[baransu]

After erlanglab/erlangpl-ui#16 PR we should cut UI 0.1 and erlangpl 0.4. Not exacly after this PR but it should be last feature for that release. It will be solid replacement for current UI and we'll cover core functionality. If you have any issues you want to include please add them to 0.4 milestone.

/ cc @michalslaski @arkgil