Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Google Test support #203

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Add Google Test support #203

wants to merge 6 commits into from

Commits on May 9, 2024

  1. Add Google Test support 

    This change adds fine grained execution of Google Test test programs.

First, the Google Test test is executed with `--gtest_list_tests`. Next,
based on the output from `--gtest_list_tests`, the testcases are run
individually. The output from each unique testcase is based on the
standard output it provides, per the test output protocol defined in
the GoogleTest docs on github [1], [2], and instrumented via various
demo programs I created, which can be found on GitHub
[here](https://github.com/ngie-eign/scratch/tree/master/programming/c%2B%2B/gtest).

This support is a very rough cut to provide an initial working
integration effort. There're additional improvements that can be made by
leveraging either the JSON or XML structured output format, instead of
scraping standard output using beginning and ending sentinels to search for
regular expressions. In order to do that though without reinventing the wheel,
Kyua would need to rely on an external JSON or XML library.

This test interface supports fine grained execution of test programs
like the ATF test interface, but matches behavior of plain/TAP interfaces by
instead supporting metadata passing via `$TEST_ENV_` prefixed environment
variables.

This support adds additional tests for verifying pass, fail, skip (will
be available in version 1.9.0 and is available in FreeBSD base's version
of Google Test), and disabled result determination, using mock output and a mock
test program (`engine/googletest_helpers`), for parity with other test formats
(ATF, plain, TAP). The helper test program purposely avoids relying on
`getopt_long*` for portability reasons, and the GoogleTest test
infrastructure, in order to limit Kyua's dependencies.

As part of this change, `store/read_transaction.cpp` needed to support
optional reasons provided with skip results. While it's bad form to omit
test results with tests, providing a reason is optional with Google
Test, and unfortunately not all portions of the test framework output a
reason when `GTEST_SKIP()` is called. See the issue in [3] for one such example
issue when `GTEST_SKIP()` is called from SetUp test fixtures.

1. https://github.com/google/googletest/blob/master/googletest/docs/primer.md
2. https://github.com/google/googletest/blob/master/googletest/docs/advanced.md
3. google/googletest#2208

Signed-off-by: Enji Cooper <yaneurabeya@gmail.com>
    @ngie-eign
    ngie-eign committed May 9, 2024
    Configuration menu
    Copy the full SHA
    b92dc4b View commit details
    Browse the repository at this point in the history

  2. Address some of @jmmv's comments in freebsd#203 

    This change does the following things:

* Leverages utils/cmdline/*, instead of reinventing the wheel.
* Consolidates googletest_results tests/setting for clarity.
* Eliminates an unnecessary variable/comments.
* Removes some duplicative TODOs/fixes TODO comment formatting.
* Inlines temporary variables where possible/sensible.
* Moves global variables into the anonymous global namespace.
* Localizes the variable initialization near its use.
* Adds more helpful comments above potentially obfuscated code.
* Use list/set initialization instead of using the equivalent unrolled
  version with `.insert()`/`.push_back()`.
* Fix indentation.

Signed-off-by: Enji Cooper <yaneurabeya@gmail.com>
    @ngie-eign
    ngie-eign committed May 9, 2024
    Configuration menu
    Copy the full SHA
    45aa670 View commit details
    Browse the repository at this point in the history

  3. Fix "make check-api-docs" with Travis CI 

    Document parameters moved into the anonymous namespace in
6e171b0 .

Signed-off-by: Enji Cooper <yaneurabeya@gmail.com>
    @ngie-eign
    ngie-eign committed May 9, 2024
    Configuration menu
    Copy the full SHA
    3e53074 View commit details
    Browse the repository at this point in the history

  4. Address more concerns made by @jmmv in freebsd#203 

    * Use initializer with std::map instead of initializing by hand.
* Fix method definition indentation.
* const poison some variables and inline where possible, eliminating
  need for unnecessary intermediate values.
* Eliminate obfuscated proposed solution in
  `engine::googletest_result::apply` and deduplicate code by using a
  lambda to check the failed vs !failed case when checking the exit
  status.
* Remove superfluous parentheses.

Signed-off-by: Enji Cooper <yaneurabeya@gmail.com>
    @ngie-eign
    ngie-eign committed May 9, 2024
    Configuration menu
    Copy the full SHA
    3c15e28 View commit details
    Browse the repository at this point in the history

  5. Provide a bogus result (a newline) when a skipped result has no context 

    This allows kyua to function in a backwards compatible manner, allowing
previous versions to read databases from newer versions.

Signed-off-by: Enji Cooper <yaneurabeya@gmail.com>
    @ngie-eign
    ngie-eign committed May 9, 2024
    Configuration menu
    Copy the full SHA
    85ee53f View commit details
    Browse the repository at this point in the history

  6. Workaround a Doxygen bug 

    The version of Doxygen used by Travis CI thinks that the global
`std::regex` variables need to have a `\return` type annotation (they
don't). Make them local to the function again and add a TODO.

Signed-off-by: Enji Cooper <yaneurabeya@gmail.com>
    @ngie-eign
    ngie-eign committed May 9, 2024
    Configuration menu
    Copy the full SHA
    b99bf61 View commit details
    Browse the repository at this point in the history