Skip to content

Conversation

@vector-of-bool
Copy link
Contributor

Refer: CDRIVER-6107

This changeset introduces the ability to apply label/tags to test cases, and exposes those as CTest labels.

This also includes new general mutable string and array/vector handling utilities, and refactors a lot of our test suite to use those.

Changes:

  • 6d908aa supports a because clause on all mlib_check assertions. This message is included in the output when the program terminates, and acts as inline documentation for why a particular assertion is present.
  • d636e33 Brings mstr, the data-owning counterpart to mstr_view. This piece has been in a stash for a while, and was brought out as a better way to handle mutable strings.
  • 9675ae5 Adds vec.t.h, a generic template header that is used to define vector types. Basically:
    • #define a type T, and then #include <mlib/vec.t.h>, and it will define a T_vec type that acts as a dynamically sized contiguous array of T objects.
    • Additional macros can be defined to add copy/destroy semantics to the vector objects.
  • f97883e Adds mstr_trim for removing whitespace from string views.
  • 47629d7 Refactors a lot of our test suite code to use mtsr and vec.t.h types. Also adds mlib/str_vec.h which just defines the mstr_vec for the very common "array of strings" type.
  • b39b795 Adds support for specifying tags/labels associated with test cases:
    • The tags are specified within the same string as the test name. (There was an attempt to add a second string parameter for tags, but this was extremely cumbersome to update all of the test cases across the codebase to pass a second empty string, especially with all the different signatures for adding test cases. A future refactor may want to consolidate our many "TestSuite_Add" functions/macros.)
    • After the test case name, place one or more spaces, and then a list of [bracketed][tags].
    • To support exporting of tags for LoadTests.cmake, the --tests-cmake switch will print CMake code that defines all the test cases. I wanted to use JSON to emit the test cases, but CMake's JSON handling functionality is incredibly slow for very large JSON blobs, so directly emitting CMake code was chosen instead.
    • LoadTests.cmake will evaluate the emitted CMake code and use it to call add_test for all the test cases, as well as apply the test case labels and fixtures.
    • If a test has a tag [uses:foo], then LoadTests.cmake will add a FIXTURES_REQUIRED of mongoc/fixtures/foo. This currently only applies to the IMDS tests, but will eventually be used for other test case fixtures.

This change allows for test cases to declare any number of associated
"tags". The tags are specified after the test case name string as a
list of bracketed tags, mimicking Catch2's syntax.

The LoadTests.cmake script has been modified to apply test case's declared
tags as CTest labels. This also gives us the ability to apply test
fixture requirements granularly on only tests that declare their
requirement (via a tag).
@vector-of-bool vector-of-bool marked this pull request as ready for review October 20, 2025 23:26
@vector-of-bool vector-of-bool requested a review from a team as a code owner October 20, 2025 23:26
@vector-of-bool
Copy link
Contributor Author

@eramongodb I'm unfortunately aware of the big slowness when running tests via CTest. The issue is indeed caused by the overhead of spawning the subprocess which then needs to register all the test cases, even if it only needs one of them.

I'm hoping in a future change, after more adoption of CTest, to speed it up significantly by simplifying the startup process, but for now it runs fast enough that I don't worry about it.

re: [slow][json] tags: Those were leftover from an earlier attempt to handle timeouts and live tests. I've removed them now.

re: [timeout:N] I wanted to have a much stricter timeout on tests, and only exempt those that were really really slow, and have tests declare their own timeout in-situ, so [timeout:N] was added as a way to do that. It may be sufficient to just set the default timeout really high, but I wanted to catch tests that were slow and specifically call them out in-source. This also gives the ability to select tests that don't need a big timeout (i.e. will run very fast) with ctest --label-exclude timeout:.

Copy link
Contributor

@eramongodb eramongodb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor feedback remaining; otherwise, LGTM!

Copy link
Collaborator

@kevinAlbs kevinAlbs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the slow review. The string and vector utilities look very useful!

benchmarks

How is ctest expected to be invoked? I see a bigger difference locally:

./cmake-build/src/libmongoc/test-libmongoc --no-fork -d --match "/bson/*"         # ~1 second
ctest --test-dir cmake-build -R "^mongoc/bson/*" --output-on-failure --parallel   # ~8 seconds

Granted, ctest is not running in CI yet. So I am OK with addressing performance as future work.

* will be destroyed and the overall copy will fail.
*
* To add a trival copying function, define `VecCopyElement` to
* `VecTrivialCopyElement`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VecTrivialCopyElement appears unused in this file (aside from Intellisense supporting macros). Is this comment out-dated?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The purpose of VecTrivialCopyElement to be used by includers, like this:

#define T int
#define VecCopyElement VecTrivialCopyElement 
#include "vec.th"

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right. Thank you for the example.

Copy link
Collaborator

@kevinAlbs kevinAlbs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Comment on lines 1120 to 1122
* @brief Like `mstr_snprintf`, but accepts a `va_list` directly.
*/
mlib_printf_attribute(1, 0) static inline mstr mstr_vsnprintf(const char *format, va_list args)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* @brief Like `mstr_snprintf`, but accepts a `va_list` directly.
*/
mlib_printf_attribute(1, 0) static inline mstr mstr_vsnprintf(const char *format, va_list args)
* @brief Like `mstr_sprintf`, but accepts a `va_list` directly.
*/
mlib_printf_attribute(1, 0) static inline mstr mstr_vsprintf(const char *format, va_list args)

Fix reference to mstr_sprintf. Suggest removing n from name since there is no size argument.

* will be destroyed and the overall copy will fail.
*
* To add a trival copying function, define `VecCopyElement` to
* `VecTrivialCopyElement`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right. Thank you for the example.

@vector-of-bool vector-of-bool merged commit 48d0a48 into mongodb:master Oct 31, 2025
44 of 46 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants