Timeline Tracking #17

Hajto · 2017-03-20T16:11:54Z

Added:

epl_timeline_observer.erl - genserver that hosts timeline changes and state changes

Edited:

epl_tracer - two new ets tables, one for storing pids that are tracked and one for caching changes

To test

{ok, TS} = epl_timeline_observer:start_link(list_to_pid("<13611.385.0>")).
%% will yield timeline so far
epl_timeline_observer:timeline(TS).

Hajto · 2017-03-21T06:14:42Z

michalslaski · 2017-03-21T07:39:21Z

Thank you very much for contributing! I will review this PR asap, hopefully tonight.

BTW, how do you like lectures on discrete mathematics? ;)

baransu · 2017-03-21T07:58:03Z

I know we had conversation about this plugin but I dont really remember when and how should we display time travel debug info in our UI.

Hajto · 2017-03-21T17:52:00Z

About UI. Ideal way would be a separate pane with list of tracked timelines, and input to type in the pid manually. (That's partially why pids are compared as lists).

Very nice thing would be also a track timeline button in one of the other views in process details. The click would start trackign and move user to timeline view.

Yet another option would be a pane with all the pids in a filterable lists, user could tick interesting pids of and see their timeline on the other subpane. (left half pane list, right half timeline)

At least that's how I would imagine it.

baransu · 2017-03-21T17:57:18Z

I'll try to add think about something for it. Is it possible to have it as separate plugin? It can be good starting point for plugin system for frontend as well because I'm thinking about moving all core plugins to one "core/official" plugins monorepo at some point and we need plugin system for that.

Hajto · 2017-03-21T18:03:28Z

The problem with making it Plugin is fact that I had to severely modify already existing tracer. (At stock it gives only information about amount of messages between processes)

michalslaski · 2017-03-31T06:00:36Z

Hi @Hajto, I was thinking how we could visualise the trace information you gather and one idea is to render so-called flame graphs. It could be rendered in the 3rd level of Vizceral, when one double-clicks a process. In other words can you please review your PR and see if you gather enough information to render a flame graph? Such graphs for Erlang processes are already implemented by @proger in his eflame.

epl-vizceral plugin

baransu · 2017-04-15T17:20:26Z

I've added epl_timeline_EPL plugin serving state over WebSocket's timeline-info topic.
I'll be working on UI part of that.

/cc @michalslaski @arkgil I would really appreciate your review on my Erlang code 🙂

baransu · 2017-04-15T21:57:53Z

You can checkout this version. Basic functionality is done, only some UI improvements and of course tracker changes if required :)

/ cc @Hajto

arkgil · 2017-04-24T13:43:03Z

And to answer your question - yes, we retrieve them all, but we can't capture changing state step by step if messages come too fast. Because messages can come to process and change its state after we receive trace message but before we make sys:get_state call.

arkgil · 2017-04-24T15:08:41Z

One more question - is it possible to show whole message received by the process? I can't find a way to extend the column with messages.

baransu · 2017-04-24T15:10:03Z

No right now. I'll add this message in state's header because side panel has limited space for that.

baransu · 2017-04-24T21:58:43Z

I've moved epl_timeline_observer.erl logic into epl_timeline.erl so it's more consistent.
I'll try to retrieve observed node pid to change input method as well as full message text 🙂

arkgil · 2017-04-26T12:13:51Z

apps/epl/src/epl_timeline.erl

-    JSON = epl_json:encode(#{timelines => T}, <<"timeline-info">>),
+handle_info({data, _, Data}, State = #state{subscribers = Subs, timelines = Times}) ->
+    {Timelines, NewState} =
+        lists:foldl(fun({Pid, {P, PidList, Changes}}, {Acc1, Acc2}) ->


This anonymous function is huge, so it might be a good idea to move it to private function in the module :)

arkgil · 2017-04-26T12:14:51Z

apps/epl/src/epl_timeline.erl

-    JSON = epl_json:encode(#{timelines => T}, <<"timeline-info">>),
+handle_info({data, _, Data}, State = #state{subscribers = Subs, timelines = Times}) ->
+    {Timelines, NewState} =
+        lists:foldl(fun({Pid, {P, PidList, Changes}}, {Acc1, Acc2}) ->


Could you rename Acc1 and Acc2 so that the reader knows what do they refer to?

I believe that was already done by @baransu.

baransu · 2017-04-28T23:09:52Z

@arkgil I'm retrieving node part of pid from kernel's application_master on observed node and then using it during the add WS message. I've also split input into two separate parts for second and third part of the pid. Waiting for your feedback 😊

michalslaski · 2017-05-01T07:00:47Z

@baransu I tried the new view and I like it very much. The UX of the timeline feature is intuitive and easy to figure out.

@Hajto I tried tracing the mnesia_locker process and noticed that throughput of messages reported in the dashboard view increased to over 45,000 msg. The system is not processing any Mnesia transactions, so most of the internal traffic is between epl_tracer and mnesia_locker processes. Is it expected?

Hajto · 2017-06-12T17:24:38Z

@michalslaski I am sorry for inactivity. It is not expected to ignore those messages. I will have to investigate it more.

michalslaski · 2017-06-13T06:06:54Z

No worries, good to have you back :)

baransu · 2017-06-19T09:26:55Z

I've added resizable panes with state and messages so when messages are to long or you want to focus on state better it's now easier 🎉

@Hajto what is progress on issues with to many messages?

baransu · 2017-06-22T15:52:07Z

@michalslaski @mkacper
I've fixed mailbox arrow navigation so right now moving through large list of messages by up/down arrows should super smooth ☺️. Waiting for your review. Let me know if you have any concerns or ideas for improvements.

mkacper · 2017-06-23T13:31:21Z

@baransu moving through mailbox by up/down arrows works perfectly but if you want to see details of a message -> state pair you have to click on a particular message. Maybe it would be easier to use when you can hit enter on a highlighted message to see the details or we can just automaticly switch the detailed view when moving through the mailbox? What do you think?

baransu · 2017-06-23T13:34:50Z

It should automatically select message on which you're on. Did you run yarn (or any make rule including it) after pulling?

mkacper · 2017-06-23T13:42:27Z

First I used only yarn start but after your comment yarn && yarn start and now it works as you said. Sorry for the confusion :) Well done! 👍

baransu · 2017-06-23T13:45:34Z

@mkacper It was because of old version of react-virtualized (very effective lib for large lists, grids etc) we're using for displaying mailbox 🙂

…erialization

Hajto added 4 commits March 18, 2017 14:46

Basics for timeline debugging

0761128

Timeline management

7e8c916

Run into problem

917cee0

Timeline tracking, caching, storing pids as lists

b06f01e

baransu added discussion enhancement labels Mar 21, 2017

baransu added a commit that referenced this pull request Apr 10, 2017

Merge pull request #17 from erlanglab/epl-vizceral-plugin

a74b177

epl-vizceral plugin

baransu added 3 commits April 13, 2017 21:55

Merge remote-tracking branch 'upstream/master'

0ca55e3

Create epl_timeline

356d110

Add epl_timeline

6f4825f

baransu self-assigned this Apr 14, 2017

baransu added 3 commits April 14, 2017 18:30

Send states over WebSocket

4e4c988

Send formated epl_timeline states over WS

ceeecc6

Only track given pid once

19478f1

First version of epl-timeline UI

4377af9

baransu added 6 commits April 18, 2017 17:32

Show messages from top to bottom

1f48b53

TImeline pids list as tabs

fc0f004

Scroll tabs on overflow

ce58a9c

Add new pid input; Change state by arrows;

d418d0a

Set current timeline pid from url param

766987c

Better timeline messages performance with react-virtualized

e5a1174

Move epl_timeline_observer logic into epl_timeline

b70db3f

baransu added 2 commits April 25, 2017 09:38

Show full message content

be8e097

Remove dummy message and state

220c41b

arkgil suggested changes Apr 26, 2017

View reviewed changes

baransu added 2 commits April 26, 2017 23:24

Extract update_timeline function from handle_info

dcf1970

Use node part of pid when using pid timeline input

c86df06

Add state highlighting to epl-timeline

bebe94a

Add resizable panes to epl-timeline

55ba987

baransu changed the title ~~[WIP] Timeline Tracking~~ Timeline Tracking Jun 21, 2017

mkacper removed this from the 0.7 milestone Jun 21, 2017

baransu added this to the 0.8 milestone Jun 22, 2017

mkacper and others added 2 commits June 22, 2017 16:02

Get processes' state only when an incoming msg is not a system one

16e3cf6

Fix mailbox arrow navigation

d9c782e

baransu added 2 commits June 27, 2017 00:47

Merge branch 'master' of https://github.com/erlanglab/erlangpl

10926bc

Rewrite eplTimelie reducer to plain ES6 from immutablejs for better s…

b07ce50

…erialization

michalslaski merged commit e9ffe23 into erlanglab:master Jun 27, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Timeline Tracking #17

Timeline Tracking #17

Hajto commented Mar 20, 2017 •

edited by baransu

Hajto commented Mar 21, 2017

michalslaski commented Mar 21, 2017

baransu commented Mar 21, 2017

Hajto commented Mar 21, 2017

baransu commented Mar 21, 2017

Hajto commented Mar 21, 2017 •

edited

michalslaski commented Mar 31, 2017

baransu commented Apr 15, 2017

baransu commented Apr 15, 2017 •

edited

arkgil commented Apr 24, 2017

arkgil commented Apr 24, 2017

baransu commented Apr 24, 2017

baransu commented Apr 24, 2017

arkgil Apr 26, 2017

arkgil Apr 26, 2017

Hajto May 5, 2017

baransu commented Apr 28, 2017

michalslaski commented May 1, 2017

Hajto commented Jun 12, 2017

michalslaski commented Jun 13, 2017

baransu commented Jun 19, 2017

baransu commented Jun 22, 2017

mkacper commented Jun 23, 2017

baransu commented Jun 23, 2017

mkacper commented Jun 23, 2017

baransu commented Jun 23, 2017

Timeline Tracking #17

Timeline Tracking #17

Conversation

Hajto commented Mar 20, 2017 • edited by baransu

Hajto commented Mar 21, 2017

michalslaski commented Mar 21, 2017

baransu commented Mar 21, 2017

Hajto commented Mar 21, 2017

baransu commented Mar 21, 2017

Hajto commented Mar 21, 2017 • edited

michalslaski commented Mar 31, 2017

baransu commented Apr 15, 2017

baransu commented Apr 15, 2017 • edited

arkgil commented Apr 24, 2017

arkgil commented Apr 24, 2017

baransu commented Apr 24, 2017

baransu commented Apr 24, 2017

arkgil Apr 26, 2017

Choose a reason for hiding this comment

arkgil Apr 26, 2017

Choose a reason for hiding this comment

Hajto May 5, 2017

Choose a reason for hiding this comment

baransu commented Apr 28, 2017

michalslaski commented May 1, 2017

Hajto commented Jun 12, 2017

michalslaski commented Jun 13, 2017

baransu commented Jun 19, 2017

baransu commented Jun 22, 2017

mkacper commented Jun 23, 2017

baransu commented Jun 23, 2017

mkacper commented Jun 23, 2017

baransu commented Jun 23, 2017

Hajto commented Mar 20, 2017 •

edited by baransu

Hajto commented Mar 21, 2017 •

edited

baransu commented Apr 15, 2017 •

edited