Improve app performance #1640

jesusgn90 · 2019-07-29T16:07:03Z

This pull request depends on wazuh/wazuh#3772, do not merge this PR if wazuh/wazuh#3772 is not merged.

Changes

12f6e0f the health check controller was calling nestedResolve and ip before being executed, however, it just needs wzConfig and ip. Now the load prior to show the health check is lower than before. In addition, we are now preventing other controllers from being launching nestedResolve when the health check was not finished.
29169ec this one is a big change, every parent state calls checkStoredAPI which heavily relies on the Wazuh API. Now, and thanks to Add new function for giving information about manager and cluster configuration wazuh#3772, we've reduced from 3 API calls to just 1 API call. Also, the logic behind this method is simpler now. This affects the whole app.
a9443ac, with this change, we now have a two seconds cache for getting API entries from the .wazuh index. Sometimes, we need to make more than one API request, then fetching this information twice was redundant, now it will be faster in all the Wazuh app.
0912f4db0f167697df530fed07ace2f4ae9b9317since a lot of methods depend on our config.yml, I've added a ten seconds cache instead of parsing the file every time it's requested.
764570b Reduced sleep in health check from 0.8s to 0.3s

Impact with the above changes

Reduced first-time load, including the health check.
Reduced load time between parent states (top menu items).
Reduced load time for all the API calls.
Reduced load time for getting the app configuration.

… is ready

jesusgn90 · 2019-07-30T11:25:39Z

Performance changes between 3.9-7.2 and 3.9-7.2-improve-performance branches

Next metrics show the number of requests and the accumulative sum of all the request times.

Just for showing the results, we are assuming the requests are not parallel. Actually, most of them are parallel so the times for both branches are lower.

Not all the requests are from the Wazuh app, Kibana does some of them.

First load including health check

	3.9-7.2	3.9-7.2-improve-performance
Accumulative time sum	3.313s	0.902s
Number of requests	26	19

Result: x3.7 faster

3.9-7.2-improve-performance

en.json	GET	304	3 ms
api_server?sense_version=%40%40SENSE_VERSION&apis=es_6_0	GET	200	7 ms
apis	GET	200	11 ms	configuration	GET	200	7 ms
_find?type=index-pattern&fields=title&per_page=10000	GET	200	17 ms
_bulk_get	POST	200	9 ms
check-stored-api	POST	200	304 ms
_bulk_get	POST	200	48 ms
wazuh-alerts-3.x-*	GET	200	44 ms
wazuh-alerts-3.x-*	GET	200	47 ms
request	POST	200	11 ms
setup	GET	200	5 ms
all	GET	200	111 ms
configuration	GET	200	5 ms
_find?type=index-pattern&fields=title&per_page=10000	GET	200	17 ms
apis	GET	200	9 ms
timestamp	GET	200	7 ms
check-stored-api	POST	200	11 ms
index-patterns	GET	200	26 ms
request	POST	200	214 ms

3.9-7.2

en.json	GET	304	3 ms
api_server?sense_version=%40%40SENSE_VERSION&apis=es_6_0	GET	200	8 ms
configuration	GET	200	5 ms
configuration	GET	200	4 ms
_find?type=index-pattern&fields=title&per_page=10000	GET	200	36 ms
apis	GET	200	13 ms
apis	GET	200	30 ms
_bulk_get	POST	200	29 ms
timestamp	GET	200	18 ms
timestamp	GET	200	10 ms
check-stored-api	POST	200	889 ms
check-stored-api	POST	200	913 ms
check-stored-api	POST	200	39 ms
request	POST	200	25 ms
setup	GET	200	42 ms
_bulk_get	POST	200	50 ms
wazuh-alerts-3.x-*	GET	200	37 ms
wazuh-alerts-3.x-*	GET	200	24 ms
all	GET	200	148 ms
configuration	GET	200	10 ms
_find?type=index-pattern&fields=title&per_page=10000	GET	200	34 ms
apis	GET	200	30 ms
timestamp	GET	200	7 ms
check-stored-api	POST	200	565 ms
index-patterns	GET	200	32 ms
request	POST	200	312 ms

First load with no health check

	3.9-7.2	3.9-7.2-improve-performance
Accumulative time sum	0.966s	0.566s
Number of requests	10	10

Result: x1.7 faster

3.9-7.2-improve-performance

en.json	GET	304	3 ms
api_server?sense_version=%40%40SENSE_VERSION&apis=es_6_0	GET	200	7 ms
configuration	GET	200	4 ms
_find?type=index-pattern&fields=title&per_page=10000	GET	200	30 ms
apis	GET	200	12 ms
timestamp	GET	200	10 ms
_bulk_get	POST	200	13 ms
check-stored-api	POST	200	232 ms
index-patterns	GET	200	26 ms
request	POST	200	229 ms

3.9-7.2

en.json	GET	304	3 ms
api_server?sense_version=%40%40SENSE_VERSION&apis=es_6_0	GET	200	8 ms
configuration	GET	200	6 ms
_find?type=index-pattern&fields=title&per_page=10000	GET	200	52 ms
apis	GET	200	28 ms
_bulk_get	POST	200	39 ms
timestamp	GET	200	37 ms
check-stored-api	POST	200	519 ms
index-patterns	GET	200	38 ms
request	POST	200	236 ms

Go from Overview to Management

	3.9-7.2	3.9-7.2-improve-performance
Accumulative time sum	0.740s	0.325s
Number of requests	6	6

Result: x2.27 faster

3.9-7.2-improve-performance

configuration	GET	200	5 ms
_find?type=index-pattern&fields=title&per_page=10000	GET	200	32 ms
apis	GET	200	33 ms
timestamp	GET	200	10 ms
check-stored-api	POST	200	221 ms
index-patterns	GET	200	24 ms

3.9-7.2

configuration	GET	200	6 ms
_find?type=index-pattern&fields=title&per_page=10000	GET	200	41 ms
apis	GET	200	35 ms
timestamp	GET	200	31 ms
check-stored-api	POST	200	594 ms
index-patterns	GET	200	33 ms

Open agents list

	3.9-7.2	3.9-7.2-improve-performance
Accumulative time sum	2.415s	1.976s
Number of requests	10	9

Result: x1.2 faster

3.9-7.2-improve-performance

configuration	GET	200	3 ms
_find?type=index-pattern&fields=title&per_page=10000	GET	200	40 ms
apis	GET	200	42 ms
timestamp	GET	200	26 ms
check-stored-api	POST	200	227 ms
1564482490294	GET	200	1.28 s
wazuh-alerts-3.x-*	GET	200	66 ms
index-patterns	GET	200	63 ms
request	POST	200	229 ms

3.9-7.2

configuration	GET	200	4 ms
_find?type=index-pattern&fields=title&per_page=10000	GET	200	56 ms
apis	GET	200	50 ms
timestamp	GET	200	21 ms
check-stored-api	POST	200	455 ms
1564482490294	GET	200	1.42 s
wazuh-alerts-3.x-*	GET	200	160 ms
index-patterns	GET	200	97 ms
wazuh-alerts-3.x-*	GET	200	54 ms
request	POST	200	258 ms

Open an agent welcome screen

	3.9-7.2	3.9-7.2-improve-performance
Accumulative time sum	1.034s	0.876s
Number of requests	8	8

Result: x1.18 faster

3.9-7.2-improve-performance

configuration	GET	200	13 ms
_find?type=index-pattern&fields=title&per_page=10000	GET	200	22 ms
apis	GET	200	7 ms
timestamp	GET	200	9 ms
check-stored-api	POST	200	233 ms
request	POST	200	280 ms
index-patterns	GET	200	47 ms
request	POST	200	265 ms

3.9-7.2

configuration	GET	200	4 ms
_find?type=index-pattern&fields=title&per_page=10000	GET	200	31 ms
apis	GET	200	7 ms
timestamp	GET	200	27 ms
check-stored-api	POST	200	425 ms
request	POST	200	241 ms
index-patterns	GET	200	57 ms
request	POST	200	242 ms

Open an agent inventory

	3.9-7.2	3.9-7.2-improve-performance
Accumulative time sum	6.700s	6.234s
Number of requests	7	7

Result: x1.07 faster

3.9-7.2-improve-performance

request	POST	200	1.18 s
request	POST	200	1.20 s
request	POST	200	1.09 s
request	POST	200	1.03 s
request	POST	200	1.02 s
001	GET	200	480 ms
request	POST	200	234 ms

3.9-7.2

request	POST	200	217 ms
001	GET	200	463 ms
request	POST	200	1.12 s
request	POST	200	1.14 s
request	POST	200	1.17 s
request	POST	200	1.29 s
request	POST	200	1.30 s

Jesús Ángel added 6 commits July 29, 2019 17:51

Reduced sleep from 0.8s to 0.3s

764570b

Reduced health-check load, prevent executing other resolves until all…

12f6e0f

… is ready

Improved checkStoredAPI method performance

29169ec

Added 2 seconds cache for "getWazuhConfigurationById"

a9443ac

Added 10 seconds cache for "getConfiguration"

0912f4d

Added "api-count" so the health check can manage API entries counter

81523df

Jesús Ángel added 10 commits July 30, 2019 15:05

Fill API cookie if needed

1cf7195

Fix condition for updating entry state

af86250

Bump code

4026091

Merge branch '3.9-7.2' into 3.9-7.2-improve-performance

1d45bfc

Executed prettier

4b0d26f

Merge branch '3.9-7.2' into 3.9-7.2-improve-performance

f2d1de8

Updated CHANGELOG

fdede3a

Bump revision (0527)

ee1221e

Updated revision in CHANGELOG

8c6d837

Updated README

5eaf3d0

jesusgn90 marked this pull request as ready for review July 31, 2019 09:38

jesusgn90 merged commit de0e46a into 3.9-7.2 Jul 31, 2019

jesusgn90 deleted the 3.9-7.2-improve-performance branch July 31, 2019 09:38

jesusgn90 mentioned this pull request Jul 31, 2019

Testing for PR #1640 #1642

Closed

11 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve app performance #1640

Improve app performance #1640

jesusgn90 commented Jul 29, 2019

jesusgn90 commented Jul 30, 2019

Improve app performance #1640

Improve app performance #1640

Conversation

jesusgn90 commented Jul 29, 2019

Changes

Impact with the above changes

jesusgn90 commented Jul 30, 2019

Performance changes between 3.9-7.2 and 3.9-7.2-improve-performance branches

First load including health check

First load with no health check

Go from Overview to Management

Open agents list

Open an agent welcome screen

Open an agent inventory