feat(performance-tests): Add k6 integration to CI pipeline jv

[JoukoVirtanen]

Description

Adds K6 load testing to CI. K6 enables developers to measure how long API calls take and how reliable they are. We need to run consistent load tests, reasonably frequently, so that we can make sure that our changes don't degrade performance of API calls. The K6 load testing is done only when the ci-performace-tests label is used.

This is branched off of #7888

User-facing documentation

• CHANGELOG is updated OR update is not needed
• documentation PR is created and is linked above OR is not needed

Testing and quality

• the change is production ready: the change is GA or otherwise the functionality is gated by a feature flag
• CI results are inspected

Automated testing

• added performance tests to CI pipeline

How I validated my change

• re-run test 2-3 times to ensure that created cluster is properly re-used
• validate that test run has expected number of generated deployments (by default: 25000)
• opensearch-json-import.py - works locally and JSON generated by test in PR can be ingested into Elastic

[rhacs-bot]

Images are ready for the commit at 26de74e.

To use with deploy scripts, first export MAIN_IMAGE_TAG=4.7.x-602-g26de74e9e5.

[github-actions]

Performance test results

Summary

  █ main dashboard

  data_received..................: 2.5 MB 294 kB/s
  data_sent......................: 158 kB 19 kB/s
  group_duration.................: avg=169.82ms min=159.92ms med=164.58ms max=313.4ms  p(90)=178.95ms p(95)=182.41ms
  http_req_blocked...............: avg=89.87µs  min=290ns    med=411ns    max=26.81ms  p(90)=541ns    p(95)=602ns   
  http_req_connecting............: avg=44.13µs  min=0s       med=0s       max=13.23ms  p(90)=0s       p(95)=0s      
  http_req_duration..............: avg=28.13ms  min=16.37ms  med=21.13ms  max=98.18ms  p(90)=57.62ms  p(95)=59.44ms 
    { expected_response:true }...: avg=28.13ms  min=16.37ms  med=21.13ms  max=98.18ms  p(90)=57.62ms  p(95)=59.44ms 
  ✓ { lib:true }.................: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
  http_req_failed................: 0.00%  ✓ 0         ✗ 300
  http_req_receiving.............: avg=236.88µs min=22.12µs  med=70.92µs  max=25.28ms  p(90)=449.12µs p(95)=524.94µs
  http_req_sending...............: avg=76.04µs  min=40.32µs  med=74.56µs  max=237.56µs p(90)=93.82µs  p(95)=100.8µs 
  http_req_tls_handshaking.......: avg=44.26µs  min=0s       med=0s       max=13.27ms  p(90)=0s       p(95)=0s      
  http_req_waiting...............: avg=27.82ms  min=16.26ms  med=20.73ms  max=98.04ms  p(90)=57.45ms  p(95)=59.3ms  
  http_reqs......................: 300    35.312498/s
  iteration_duration.............: avg=169.89ms min=160ms    med=164.64ms max=313.64ms p(90)=179.01ms p(95)=182.53ms
  iterations.....................: 50     5.885416/s
  vus............................: 1      min=1       max=1
  vus_max........................: 1      min=1       max=1

Sources

• Summary: HTML Report | JSON
• Raw Data: CSV | JSON

[mtodor]

Tnx for making progress on this! I have added a few comments. I don't see much to be done, just a few more tweaks and a bit of re-testing.

Please also check styles: https://github.com/stackrox/stackrox/actions/runs/13091435913/job/36528512327?pr=14069

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 48.92%. Comparing base (0ba45f3) to head (26de74e).
Report is 43 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff            @@
##           master   #14069    +/-   ##
========================================
  Coverage   48.92%   48.92%            
========================================
  Files        2500     2500            
  Lines      180852   180982   +130     
========================================
+ Hits        88479    88545    +66     
- Misses      85353    85412    +59     
- Partials     7020     7025     +5     

Flag	Coverage Δ
go-unit-tests	48.92% <ø> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[JoukoVirtanen]

/test ocp-4-17-nongroovy-e2e-tests

[mtodor]

/test ocp-4-17-nongroovy-e2e-tests

[mtodor]

@JoukoVirtanen Tnx a lot for continuing on this!

I have tested python script locally. It works.

[davdhacs]

@JoukoVirtanen Tnx a lot for continuing on this!

I have tested python script locally. It works.

+1 This is great! @mtodor I created a user and added it to the github org secrets with these variable names so the python script can be added into an action run.

[tommartensen]

@davdhacs thanks for taking care of this!

[tommartensen]

@JoukoVirtanen thanks for picking this up! Will you be adding the opensearch upload to the workflow / is there a ticket to track against?