History tab very slow

[johnarnold]

The st2web History tab is very slow to interact with. For example, click to expand a workflow and see underlying actions can take 30+ seconds, sometimes much longer. Can't effectively use the search or filter fields as they are slow/unresponsive or we get unexpected behavior (click to search, panel reloads).

Stackstorm 2.9.0, Ubuntu 16.04, Chrome 60.0.3112.78

We have a scaled out deployment with (4) VMs, 1PPC docker containers, ~9 worker processes for gunicorn stuff (st2web/st2auth/st2api). Overall utilization on the VMs is very low <10%

Using chrome's Developer Tools, I observe that it spends the most time on loading things from /api/v1/executions

[Kami]

We included some of the improvements for that in v2.9.1 release - #605.

Can you please try with st2web v2.9.1 and report back?

[johnarnold]

@Kami I upgraded to 2.9.1 over the weekend. Still slow.

root@45208c6fa393:/# st2 --version
st2 2.9.1, on Python 2.7.6

[Kami]

@johnarnold Can you please provide some screenshots of the Chrome network tab so we can see what is going on?

It would also help to know how big are the executions which take a long time to load - is the result large? How large?

Thanks

[Kami]

To add some context - improvements we did refer to the initial history page load. There we only retrieve fields we need and that should be much faster in StackStorm >= v2.9.0.

Another problematic view is execution "drill down" view aka when user clicks on an execution to expand it.

We should perform the same include attributes filtering there and only retrieve "result" field once user clicks on a specific execution and that execution is actually displayed in the right tab (and we should probably only do this for child executions and not massive Mistral workflow executions which contains all child data and could be massive).

/cc @VineeshJain

[Kami]

There are still some edge cases where we try to render large results which can freeze the browser window, but #868 and related changes should mostly fix this issue.