Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix loadgenerator race condition #1029

Merged

Conversation

TheRealSpaceShip
Copy link
Contributor

Handle to avoid calls of authenticated endpoints when logout has already been called

Background

Almost 50% of authenticated requests are failing with error code 401 because of the request-logout race.

Change Summary

Fixed logic on how signup+login and logout works. Also made some small improvements.

@TheRealSpaceShip TheRealSpaceShip requested a review from a team as a code owner October 17, 2022 11:57
@bourgeoisor
Copy link
Member

Lint is failing:

Run> pylint --rcfile=./.pylintrc ./src/*/*.py
************* Module locustfile
src/loadgenerator/locustfile.py:[5](https://github.com/GoogleCloudPlatform/bank-of-anthos/actions/runs/3264911106/jobs/5368966183#step:5:6)3:12: R1705: Unnecessary "else" after "return" (no-else-return)

@bourgeoisor bourgeoisor self-assigned this Oct 17, 2022
@TheRealSpaceShip TheRealSpaceShip force-pushed the fix-loadgenerator-race-condition branch 2 times, most recently from 9c1682b to 8ffd87c Compare October 18, 2022 08:36
@bourgeoisor
Copy link
Member

The load generator seems to be more in-line with failure expectations:

Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
GET      /                                                                               6385     0(0.00%) |     97      24    9454     35 |    1.20        0.00
POST     /deposit                                                                        3830     9(0.23%) |    382     139   13230    310 |    0.30        0.00
GET      /home                                                                           6194     0(0.00%) |     89      23    6076     33 |    1.00        0.00
GET      /login                                                                          3576    11(0.31%) |     19       2    7240      5 |    0.00        0.00
POST     /login                                                                          3124     0(0.00%) |   3227    1313   10527   3100 |    0.40        0.00
POST     /logout                                                                          635     0(0.00%) |     37       6    3824      9 |    0.00        0.00
POST     /payment                                                                        3105    16(0.52%) |    379      69    5129    310 |    0.60        0.00
GET      /signup                                                                         3483     4(0.11%) |     28       2   15380      5 |    0.00        0.00
POST     /signup                                                                          665    25(3.76%) |   4603       2   14009   4300 |    0.00        0.00
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
         Aggregated                                                                     30997    65(0.21%) |    553       2   15380     51 |    3.50        0.00

Vs. what's currently in the main branch:

Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
GET      /                                                                               7783     0(0.00%) |     48       2    8687     28 |    0.90        0.00
POST     /deposit                                                                        4623 2267(49.04%) |    193       2    8401    290 |    0.70        0.10
GET      /home                                                                           7912 3874(48.96%) |     49       4    8553     27 |    0.90        0.30
GET      /login                                                                          2008     0(0.00%) |     14       2    3254      4 |    0.20        0.00
POST     /login                                                                          3867 1859(48.07%) |   1415       2    9517   1400 |    0.40        0.00
POST     /logout                                                                          743     0(0.00%) |     22       5    1066      9 |    0.30        0.00
POST     /payment                                                                        3822 1877(49.11%) |    189       2    7293    290 |    0.20        0.00
GET      /signup                                                                         2058     0(0.00%) |     16       2    3143      5 |    0.20        0.00
POST     /signup                                                                          386    11(2.85%) |   3987    1858   11911   3600 |    0.10        0.00
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
         Aggregated                                                                     33202 9888(29.78%) |    285       2   11911     10 |    3.90        0.40

src/loadgenerator/locustfile.py Outdated Show resolved Hide resolved
src/loadgenerator/locustfile.py Outdated Show resolved Hide resolved
Handle to avoid calls of authenticated endpoints when logout has already called
@bourgeoisor
Copy link
Member

After latest commit:

Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
GET      /                                                                                433     0(0.00%) |    173      32    8063     57 |    0.40        0.00
POST     /deposit                                                                         272     0(0.00%) |    558     300    8330    350 |    0.40        0.00
GET      /home                                                                            438     0(0.00%) |    208      30    8041     58 |    0.60        0.00
GET      /login                                                                           265     0(0.00%) |     63       3    2759      5 |    0.00        0.00
POST     /login                                                                           200     0(0.00%) |   4881    1576   12607   4500 |    0.40        0.00
POST     /logout                                                                           40     0(0.00%) |     99       8    2469     10 |    0.00        0.00
POST     /payment                                                                         228     9(3.95%) |    695     304   11205    350 |    0.30        0.00
GET      /signup                                                                          245     0(0.00%) |     15       3    1062      5 |    0.00        0.00
POST     /signup                                                                           61   16(26.23%) |   5893      38   13932   4900 |    0.10        0.00
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
         Aggregated                                                                      2182    25(1.15%) |    842       3   13932     77 |    2.20        0.00

Looks great!

Copy link
Member

@bourgeoisor bourgeoisor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@bourgeoisor bourgeoisor merged commit 88aa0b8 into GoogleCloudPlatform:main Oct 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants