Skip to content

Improvement: Introduce names for test levels#1731

Merged
sandro-elsweijer merged 14 commits intomainfrom
impr_test_level_names
Jul 2, 2025
Merged

Improvement: Introduce names for test levels#1731
sandro-elsweijer merged 14 commits intomainfrom
impr_test_level_names

Conversation

@spenke91
Copy link
Collaborator

@spenke91 spenke91 commented Jul 1, 2025

Describe your changes here:

Closes #1677.

To avoid confusion, this PR changes the option T8_CODE_TEST_LEVEL to a property accepting only certain choices, i.e., T8_TEST_LEVEL_FULL, T8_TEST_LEVEL_MEDIUM, and T8_TEST_LEVEL_BASIC. In ccmake, you can thus switch through the options hitting ENTER, similar to CMAKE_BUILD_TYPE.

Note that in the background, this T8CODE_TEST_LEVEL string is translated to an integer value T8_TEST_LEVEL_INT with the same values T8CODE_TEST_LEVEL had before this PR: 0 for full tests, 1 for intermediate, 2 for basic.

Of course, I am open to change this, but I introduced it because:

  1. it simplifies distinguishing different cases in the code of the test cases a lot, e.g., allowing statements like #if T8_TEST_LEVEL_INT >= 1;
  2. it makes it easier to introduce further test levels in the future without having to change all associated #if statements

In case you're interested, another option I considered is adding a new option T8_CODE_TEST_LEVEL_NAME that allows you to specificy the test level by name, while still keeping the option of setting it as integer alive. However, i felt like that was bound to cause problems with contradicting values etc...

All these boxes must be checked by the AUTHOR before requesting review:

  • The PR is small enough to be reviewed easily. If not, consider splitting up the changes in multiple PRs.
  • The title starts with one of the following prefixes: Documentation:, Bugfix:, Feature:, Improvement: or Other:.
  • If the PR is related to an issue, make sure to link it.
  • The author made sure that, as a reviewer, he/she would check all boxes below.

All these boxes must be checked by the REVIEWERS before merging the pull request:

As a reviewer please read through all the code lines and make sure that the code is fully understood, bug free, well-documented and well-structured.

General

  • The reviewer executed the new code features at least once and checked the results manually.
  • The code follows the t8code coding guidelines.
  • New source/header files are properly added to the CMake files.
  • The code is well documented. In particular, all function declarations, structs/classes and their members have a proper doxygen documentation.
  • All new algorithms and data structures are sufficiently optimal in terms of memory and runtime (If this should be merged, but there is still potential for optimization, create a new issue).

Tests

  • The code is covered in an existing or new test case using Google Test.
  • The code coverage of the project (reported in the CI) should not decrease. If coverage is decreased, make sure that this is reasonable and acceptable.
  • Valgrind doesn't find any bugs in the new code. This script can be used to check for errors; see also this wiki article.

If the Pull request introduces code that is not covered by the github action (for example coupling with a new library):

  • Should this use case be added to the github action?
  • If not, does the specific use case compile and all tests pass (check manually).

Scripts and Wiki

  • If a new directory with source files is added, it must be covered by the script/find_all_source_files.scp to check the indentation of these files.
  • If this PR introduces a new feature, it must be covered in an example or tutorial and a Wiki article.

License

  • The author added a BSD statement to doc/ (or already has one).

@spenke91 spenke91 marked this pull request as ready for review July 1, 2025 12:31
Copy link
Member

@sandro-elsweijer sandro-elsweijer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice that you are considering to work on our CI! It is an honor and responsibility which you will keep forever ;)
Only one small remark :)

@spenke91
Copy link
Collaborator Author

spenke91 commented Jul 1, 2025

@sandro-elsweijer Sorry for the many github mails ;-) Something is off with the workflow now, it seems like it does not accept the type: choice or rather the options:. I will have a more concentrated look at it when it is less hot at my desk ;-)

@spenke91 spenke91 assigned spenke91 and unassigned sandro-elsweijer Jul 1, 2025
@spenke91
Copy link
Collaborator Author

spenke91 commented Jul 2, 2025

As much as I liked the idea of using type: choice for the workflows, it does not seem to work. According to github/vscode-github-actions#318, it is only supported for inputs of on.workflow_dispatch event, not on.workflow_call.
So I changed it back to type: string and it should work again (soon) 👍

@codecov
Copy link

codecov bot commented Jul 2, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 74.08%. Comparing base (640e70f) to head (a44a565).
Report is 48 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1731   +/-   ##
=======================================
  Coverage   74.08%   74.08%           
=======================================
  Files          99       99           
  Lines       18619    18619           
=======================================
  Hits        13793    13793           
  Misses       4826     4826           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@spenke91 spenke91 assigned sandro-elsweijer and unassigned spenke91 Jul 2, 2025
@sandro-elsweijer
Copy link
Member

As much as I liked the idea of using type: choice for the workflows, it does not seem to work. According to github/vscode-github-actions#318, it is only supported for inputs of on.workflow_dispatch event, not on.workflow_call. So I changed it back to type: string and it should work again (soon) 👍

One has to love to work on github workflows

@sandro-elsweijer sandro-elsweijer added this pull request to the merge queue Jul 2, 2025
Merged via the queue into main with commit d06d4ee Jul 2, 2025
24 checks passed
@sandro-elsweijer sandro-elsweijer deleted the impr_test_level_names branch July 2, 2025 09:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Assign names to the test levels instead of numbers

2 participants