Skip to content

Split rest_api test runs to improve performance #9550

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 25 commits into from
Jun 26, 2025

Conversation

archibald1418
Copy link
Contributor

@archibald1418 archibald1418 commented Jun 19, 2025

This splits rest_api test execution between two groups of tests executed on two different github-runners.
rest_api_testing job now consists of 3 worker: slow batch with cache, slow batch no cache, fast batch
Runtime reduced from 50min to 25min

The two slowest test classes tests/python/rest_api/test_tasks.py::{TestTaskData, TestPostTaskData} are now run separately and parametrized by CVAT_ALLOW_STATIC_CACHE server parameter.

Allure reports and coverage report routines were updated accordingly, similar to e2e_testing github workflow job

How has this been tested?

This is a test refactoring PR, so the only test for this is that tests don't crash or fail unexpectedly. The three groups are executed separately each using their own instance of the current testing infra

Checklist

  • I submit my changes into the develop branch
  • I have created a changelog fragment
  • I have updated the documentation accordingly
  • I have added tests to cover my changes
  • I have linked related issues (see GitHub docs)

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.

@archibald1418 archibald1418 marked this pull request as ready for review June 19, 2025 16:24
@archibald1418 archibald1418 changed the title Split rest_api test runs in two Split rest_api test runs in two to improve performance Jun 19, 2025
@archibald1418 archibald1418 force-pushed the ov/tests-rest-api-sharding branch from 553dc63 to 415de02 Compare June 20, 2025 00:37
@codecov-commenter
Copy link

codecov-commenter commented Jun 20, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.74%. Comparing base (dcccc44) to head (7d05d90).

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #9550      +/-   ##
===========================================
- Coverage    73.78%   71.74%   -2.05%     
===========================================
  Files          448      441       -7     
  Lines        46774    46246     -528     
  Branches      3943     3943              
===========================================
- Hits         34514    33178    -1336     
- Misses       12260    13068     +808     
Components Coverage Δ
cvat-ui 77.74% <ø> (+0.01%) ⬆️
cvat-server 66.99% <ø> (-3.74%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@archibald1418 archibald1418 requested a review from orazvaliev June 23, 2025 11:25
@archibald1418 archibald1418 requested a review from SpecLad June 23, 2025 11:25
@archibald1418 archibald1418 removed the request for review from SpecLad June 23, 2025 12:30
@archibald1418
Copy link
Contributor Author

TODO: add license on new files

Copy link

Quality Gate Failed Quality Gate failed

Failed conditions
6.1% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

@archibald1418 archibald1418 requested a review from orazvaliev June 26, 2025 15:47
@archibald1418 archibald1418 merged commit 45dd0c1 into develop Jun 26, 2025
36 of 37 checks passed
@archibald1418 archibald1418 deleted the ov/tests-rest-api-sharding branch June 26, 2025 18:03
@archibald1418 archibald1418 changed the title Split rest_api test runs in two to improve performance Split rest_api test runs to improve performance Jun 27, 2025
@archibald1418
Copy link
Contributor Author

/check

Copy link
Contributor

github-actions bot commented Jun 27, 2025

🚫 Workflows has been canceled
📄 See logs here

@archibald1418
Copy link
Contributor Author

/check

Copy link
Contributor

github-actions bot commented Jun 27, 2025

✔️ All checks completed successfully
📄 See logs here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants