Update tile load test script and add scripts for hompage load, facility download, and facility POST #1777

jwalgran · 2022-03-26T05:29:10Z

Overview

Update tile load test script and add scripts for hompage load, facility download, and facility POST

Connects #1776

Testing Instructions

NOTE these scripts have been successfully used on staging so this can be a cursory review.

Register an account on staging and generate an API token
Obtain the CLIENT_KEY used on staging
cd into the load-tests directory.
Verify that you can run the new load test commands added to the README in the PR

Checklist

fixup! commits have been squashed
CI passes after rebase
CHANGELOG.md updated with summary of features or fixes, following Keep a Changelog guidelines

jwalgran · 2022-03-26T06:12:54Z

load-tests/browse_homepage.js

+            buildGet(`${rootUrl}/static/css/2.037bc208.chunk.css`),
+            buildGet(`${rootUrl}/static/css/main.cfc7f5ef.chunk.css`),
+            buildGet(`${rootUrl}/static/js/2.b90b8892.chunk.js`),
+            buildGet(`${rootUrl}/static/js/main.058b1d83.chunk.js`),


These are going to be different for each release. I think the way to get these dynamically is to search the HTML returned from a request to /

emilyhu0106

The tests looks very good! Wondering if below is the expected output for the facility POSTing test?

  execution: local
     script: /scripts/post_facility.js
     output: -

  scenarios: (100.00%) 1 scenario, 5 max VUs, 10m30s max duration (incl. graceful stop):
           * default: 5000 iterations shared among 5 VUs (maxDuration: 10m0s, gracefulStop: 30s)


running (03m34.3s), 0/5 VUs, 5000 complete and 0 interrupted iterations
default ✓ [======================================] 5 VUs  03m34.6s/10m0s  5000/5000 shared iters

     ✗ is status 2xx
      ↳  0% — ✓ 0 / ✗ 5000

     checks.....................: 0.00%   ✓ 0         ✗ 5000
     data_received..............: 2.0 MB  9.2 kB/s
     data_sent..................: 1.3 MB  6.2 kB/s
     http_req_blocked...........: avg=388.49µs min=25.3µs   med=35.4µs   max=342.4ms  p(90)=76.71µs  p(95)=113.3µs 
     http_req_connecting........: avg=88.7µs   min=0s       med=0s       max=92.85ms  p(90)=0s       p(95)=0s      
     http_req_duration..........: avg=213.29ms min=144.79ms med=199.96ms max=4.64s    p(90)=232.73ms p(95)=254.86ms
     http_req_failed............: 100.00% ✓ 5000      ✗ 0   
     http_req_receiving.........: avg=301.9µs  min=69.3µs   med=192.5µs  max=22.68ms  p(90)=439.34µs p(95)=664.28µs
     http_req_sending...........: avg=326.84µs min=118.7µs  med=287.7µs  max=1.8ms    p(90)=496.43µs p(95)=603.02µs
     http_req_tls_handshaking...: avg=134.93µs min=0s       med=0s       max=138.33ms p(90)=0s       p(95)=0s      
     http_req_waiting...........: avg=212.66ms min=143.8ms  med=199.35ms max=4.64s    p(90)=232.06ms p(95)=254.3ms 
     http_reqs..................: 5000    23.327565/s
     iteration_duration.........: avg=214.48ms min=179.94ms med=200.66ms max=4.98s    p(90)=233.45ms p(95)=255.38ms
     iterations.................: 5000    23.327565/s
     vus........................: 5       min=5       max=5 
     vus_max....................: 5       min=5       max=5

I also did some readings about k6 and load tests, reviewing this PR is helpful!

jwalgran · 2022-05-04T14:35:44Z

Sorry for the delayed response on your comment. The POST test output your shared shows that all the requests responded with a status other than 200, which is not correct.

That script does require that TOKEN is set to a valid API token from staging. Did you run the command with a token from a staging account? If not, I can share my staging token with you that is configured for unlimited staging API access

 TOKEN=abc123-some-real-token-here \
docker-compose -f docker-compose.yml \
  run --rm k6 run -u 5 -i 5000  /scripts/post_facility.js

emilyhu0106 · 2022-05-04T19:53:56Z

I just ran this test twice with a staging API Token.

The first time failed in the middle of the tests, probably due to api limits. The result is as below:

running (10m02.3s), 0/5 VUs, 1463 complete and 0 interrupted iterations
default ✗ [==========>---------------------------] 5 VUs  10m03.0s/10m0s  1463/5000 shared iters

     ✗ is status 2xx
      ↳  96% — ✓ 1405 / ✗ 58

Then I gave myself unlimited api calls, ran it again. It was complete but failed

running (03m09.4s), 0/5 VUs, 5000 complete and 0 interrupted iterations
default ✓ [======================================] 5 VUs  03m09.6s/10m0s  5000/5000 shared iters

     ✗ is status 2xx
      ↳  0% — ✓ 0 / ✗ 5000

Should I try with another API token?

jwalgran · 2022-05-05T22:32:05Z

I clicked the "close" button by accident.

jwalgran · 2022-05-05T22:40:49Z

I scheduled a pairing session on Friday May 6 to look at this together.

emilyhu0106

All tests are passed after the pairing. It was related to the API blocks that can be updated in the Admin Dashboard. Thank you Justin!

Allows for testing tile requests for different facility filters, some of which may be more expensive than others.

Ensures that the requests in parallel runs using VU_MULTIPLIER do not line up. Attempting to make a more realistic test.

The tile load test uses virtual users and iterations to chunk and run requests read from a HAR file and exists after one set of tile requests is done. This helper loops a run of the scripts with a different CACHE_KEY_SUFFIX for each run to allow for testing sustained continuous load.

This script was initially created by browsing the home page, exporting the requests as a HAR file, converting the HAR file to a script using k6.io, then modifying the script to simulate the parallel request behavior of a browser. Making API requests requires a CLIENT_KEY so we update the load testing compose file to pass through an appropriate environment variable.

Mimics the process used by the application to load a page of facilities at a time. Calling the API requires setting the CLIENT_KEY environment variable. FILTERS can be set to change the query string arguments passed to each page request.

Includes definitions of all supported env vars and examples of how to run each of the load test scripts.

To simulate API partners POSTing a list of facilities to the API. `addresses.csv` was generated by downloading facility addresses from the staging data set. Portions of the random-ish facility name generator were adapted from https://github.com/joke2k/faker/blob/5b98d2ab877e414b51e76abc8fdde2fb2dad6710/faker/providers/company/__init__.py

jwalgran · 2022-05-20T20:17:23Z

Thanks for the review.

jwalgran commented Mar 26, 2022

View reviewed changes

jwalgran changed the title ~~Update tile load test script and add scripts for hompage load and facility download~~ Update tile load test script and add scripts for hompage load, facility download, and facility POST Mar 28, 2022

jwalgran requested a review from emilyhu0106 April 29, 2022 19:18

jwalgran assigned emilyhu0106 Apr 29, 2022

jwalgran marked this pull request as ready for review April 29, 2022 19:18

emilyhu0106 reviewed Apr 30, 2022

View reviewed changes

jwalgran closed this May 5, 2022

jwalgran reopened this May 5, 2022

emilyhu0106 approved these changes May 9, 2022

View reviewed changes

jwalgran added 8 commits May 20, 2022 10:54

Allow setting filters for tile load testing

3690ee9

Allows for testing tile requests for different facility filters, some of which may be more expensive than others.

Insert a small random sleep into tile load test requests

e96a8a8

Ensures that the requests in parallel runs using VU_MULTIPLIER do not line up. Attempting to make a more realistic test.

Add a facility download load test script

6b667b4

Mimics the process used by the application to load a page of facilities at a time. Calling the API requires setting the CLIENT_KEY environment variable. FILTERS can be set to change the query string arguments passed to each page request.

Update load testing README

09b6414

Includes definitions of all supported env vars and examples of how to run each of the load test scripts.

Update CHANGELOG

bbd1dc9

jwalgran force-pushed the feature/jcw/load-testing branch from 3125570 to bbd1dc9 Compare May 20, 2022 17:58

jwalgran merged commit a4e6040 into develop May 20, 2022

jwalgran mentioned this pull request May 20, 2022

Update tile load test script and add scripts for hompage load, facility download, and facility POST #1857

Merged

5 tasks

jwalgran deleted the feature/jcw/load-testing branch September 20, 2022 19:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update tile load test script and add scripts for hompage load, facility download, and facility POST #1777

Update tile load test script and add scripts for hompage load, facility download, and facility POST #1777

jwalgran commented Mar 26, 2022 •

edited

jwalgran Mar 26, 2022

emilyhu0106 left a comment

jwalgran commented May 4, 2022

emilyhu0106 commented May 4, 2022

jwalgran commented May 5, 2022

jwalgran commented May 5, 2022

emilyhu0106 left a comment

jwalgran commented May 20, 2022

Update tile load test script and add scripts for hompage load, facility download, and facility POST #1777

Update tile load test script and add scripts for hompage load, facility download, and facility POST #1777

Conversation

jwalgran commented Mar 26, 2022 • edited

Overview

Testing Instructions

Checklist

jwalgran Mar 26, 2022

Choose a reason for hiding this comment

emilyhu0106 left a comment

Choose a reason for hiding this comment

jwalgran commented May 4, 2022

emilyhu0106 commented May 4, 2022

jwalgran commented May 5, 2022

jwalgran commented May 5, 2022

emilyhu0106 left a comment

Choose a reason for hiding this comment

jwalgran commented May 20, 2022

jwalgran commented Mar 26, 2022 •

edited