-
Notifications
You must be signed in to change notification settings - Fork 456
[CDRIVER-6107] Add Test Case Tag Support #2135
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CDRIVER-6107] Add Test Case Tag Support #2135
Conversation
This change allows for test cases to declare any number of associated "tags". The tags are specified after the test case name string as a list of bracketed tags, mimicking Catch2's syntax. The LoadTests.cmake script has been modified to apply test case's declared tags as CTest labels. This also gives us the ability to apply test fixture requirements granularly on only tests that declare their requirement (via a tag).
|
@eramongodb I'm unfortunately aware of the big slowness when running tests via CTest. The issue is indeed caused by the overhead of spawning the subprocess which then needs to register all the test cases, even if it only needs one of them. I'm hoping in a future change, after more adoption of CTest, to speed it up significantly by simplifying the startup process, but for now it runs fast enough that I don't worry about it. re: re: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor feedback remaining; otherwise, LGTM!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for the slow review. The string and vector utilities look very useful!
benchmarks
How is ctest expected to be invoked? I see a bigger difference locally:
./cmake-build/src/libmongoc/test-libmongoc --no-fork -d --match "/bson/*" # ~1 second
ctest --test-dir cmake-build -R "^mongoc/bson/*" --output-on-failure --parallel # ~8 seconds
Granted, ctest is not running in CI yet. So I am OK with addressing performance as future work.
| * will be destroyed and the overall copy will fail. | ||
| * | ||
| * To add a trival copying function, define `VecCopyElement` to | ||
| * `VecTrivialCopyElement`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
VecTrivialCopyElement appears unused in this file (aside from Intellisense supporting macros). Is this comment out-dated?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The purpose of VecTrivialCopyElement to be used by includers, like this:
#define T int
#define VecCopyElement VecTrivialCopyElement
#include "vec.th"There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah right. Thank you for the example.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
src/common/src/mlib/str.h
Outdated
| * @brief Like `mstr_snprintf`, but accepts a `va_list` directly. | ||
| */ | ||
| mlib_printf_attribute(1, 0) static inline mstr mstr_vsnprintf(const char *format, va_list args) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| * @brief Like `mstr_snprintf`, but accepts a `va_list` directly. | |
| */ | |
| mlib_printf_attribute(1, 0) static inline mstr mstr_vsnprintf(const char *format, va_list args) | |
| * @brief Like `mstr_sprintf`, but accepts a `va_list` directly. | |
| */ | |
| mlib_printf_attribute(1, 0) static inline mstr mstr_vsprintf(const char *format, va_list args) |
Fix reference to mstr_sprintf. Suggest removing n from name since there is no size argument.
| * will be destroyed and the overall copy will fail. | ||
| * | ||
| * To add a trival copying function, define `VecCopyElement` to | ||
| * `VecTrivialCopyElement`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah right. Thank you for the example.
Refer: CDRIVER-6107
This changeset introduces the ability to apply label/tags to test cases, and exposes those as CTest labels.
This also includes new general mutable string and array/vector handling utilities, and refactors a lot of our test suite to use those.
Changes:
becauseclause on allmlib_checkassertions. This message is included in the output when the program terminates, and acts as inline documentation for why a particular assertion is present.mstr, the data-owning counterpart tomstr_view. This piece has been in a stash for a while, and was brought out as a better way to handle mutable strings.vec.t.h, a generic template header that is used to define vector types. Basically:#definea typeT, and then#include <mlib/vec.t.h>, and it will define aT_vectype that acts as a dynamically sized contiguous array ofTobjects.mstr_trimfor removing whitespace from string views.mtsrandvec.t.htypes. Also addsmlib/str_vec.hwhich just defines themstr_vecfor the very common "array of strings" type.[bracketed][tags].LoadTests.cmake, the--tests-cmakeswitch will print CMake code that defines all the test cases. I wanted to use JSON to emit the test cases, but CMake's JSON handling functionality is incredibly slow for very large JSON blobs, so directly emitting CMake code was chosen instead.LoadTests.cmakewill evaluate the emitted CMake code and use it to calladd_testfor all the test cases, as well as apply the test case labels and fixtures.[uses:foo], thenLoadTests.cmakewill add aFIXTURES_REQUIREDofmongoc/fixtures/foo. This currently only applies to the IMDS tests, but will eventually be used for other test case fixtures.