Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ping performance test #2337

Merged
merged 22 commits into from
May 18, 2021

Conversation

sanderegg
Copy link
Member

@sanderegg sanderegg commented May 18, 2021

What do these changes do?

  • improves usage of the locust.io perfomance tool
  • created an aggressive test for pinging deployments

the test simulates 100 users calling 10x the root of the osparc platform and 1x the /v0/health endpoint for 1 minute.
the test fails if the average response time is above 200ms and if the failure rate is above 1%

my current statistics

  • local: 0% failure, the average response time: 44ms with a max at 1098ms, doing about 2122 requests/seconds in total.
  • master: 0% failure, the average response time: 79ms with a max at 2124ms, doing about 1198 requests/seconds in total.

Related issue/s

How to test

Checklist

@sanderegg sanderegg added a:infra+ops maintenance of infrastructure or operations (discussed in retro) t:maintenance Some planned maintenance work labels May 18, 2021
@sanderegg sanderegg added this to the Chinchilla milestone May 18, 2021
@sanderegg sanderegg self-assigned this May 18, 2021
check for avg response below 200ms
@codecov
Copy link

codecov bot commented May 18, 2021

Codecov Report

Merging #2337 (a0b94a3) into master (d9beaae) will decrease coverage by 12.9%.
The diff coverage is n/a.

❗ Current head a0b94a3 differs from pull request most recent head f5b368e. Consider uploading reports for the commit f5b368e to get more accurate results
Impacted file tree graph

@@            Coverage Diff            @@
##           master   #2337      +/-   ##
=========================================
- Coverage    63.4%   50.5%   -13.0%     
=========================================
  Files         485     277     -208     
  Lines       18104   10725    -7379     
  Branches     1740     943     -797     
=========================================
- Hits        11485    5418    -6067     
+ Misses       6286    5215    -1071     
+ Partials      333      92     -241     
Flag Coverage Δ
unittests 50.5% <ø> (-12.9%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...es/web/server/src/simcore_service_webserver/log.py 0.0% <0.0%> (-68.0%) ⬇️
...server/src/simcore_service_webserver/groups_api.py 21.7% <0.0%> (-67.4%) ⬇️
.../simcore_service_webserver/projects/projects_db.py 19.9% <0.0%> (-66.6%) ⬇️
...re_service_webserver/projects/projects_handlers.py 26.0% <0.0%> (-63.8%) ⬇️
...er/src/simcore_service_webserver/studies_access.py 27.3% <0.0%> (-61.4%) ⬇️
...mcore_service_webserver/login/api_keys_handlers.py 36.7% <0.0%> (-55.9%) ⬇️
..._service_webserver/studies_dispatcher/_projects.py 34.0% <0.0%> (-54.0%) ⬇️
...src/simcore_service_webserver/socketio/handlers.py 32.9% <0.0%> (-49.5%) ⬇️
...src/simcore_service_webserver/groups_exceptions.py 53.3% <0.0%> (-46.7%) ⬇️
...r/src/simcore_service_webserver/socketio/events.py 54.5% <0.0%> (-45.5%) ⬇️
... and 251 more

@sanderegg sanderegg requested a review from Surfict May 18, 2021 12:33
Comment on lines 20 to 28
## Usage in CI (headless mode)

```console
# builds the distributed locust worker/master image
make build
# runs the test defined by locust_files/platform_ping_test.py going to http://127.0.0.1:8089 allows to see the UI
make test target=platform_ping_test.py
# removes the containers
make down
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not see the difference with the above # Usage paragraph

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make up vs make test

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Above is up and below is test

tests/performance/Makefile Outdated Show resolved Hide resolved
tests/performance/Dockerfile Outdated Show resolved Hide resolved
@sanderegg sanderegg requested a review from pcrespov May 18, 2021 13:19
tests/performance/README.md Outdated Show resolved Hide resolved
Comment on lines 20 to 28
## Usage in CI (headless mode)

```console
# builds the distributed locust worker/master image
make build
# runs the test defined by locust_files/platform_ping_test.py going to http://127.0.0.1:8089 allows to see the UI
make test target=platform_ping_test.py
# removes the containers
make down
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Above is up and below is test

@sanderegg sanderegg merged commit cb2fef9 into ITISFoundation:master May 18, 2021
@sanderegg sanderegg deleted the add_ping_performance_test branch May 18, 2021 13:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:infra+ops maintenance of infrastructure or operations (discussed in retro) t:maintenance Some planned maintenance work
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants