Consider renaming term "test case" to "test suite"

[ghost]

Google Test is using the term "test case" as sort of container of "test", as is written in Primer.md:

A test case contains one or many tests. You should group your tests into test cases that reflect the structure of the tested code. When multiple tests in a test case need to share common objects and subroutines, you can put them into a test fixture class.

It seems to me, that contrary to this use of the term "test case", most quality assurance literature including the International Software Testing Qualifications Board (ISTQB) are using the term "test suite" for this purpose. E.g. ISTQB's glossary about test suites:

A set of several test cases for a component or system under test, where the post condition of one test is often used as the precondition for the next one.

And about test case:

A set of input values, execution preconditions, expected results and execution postconditions, developed for a particular objective or test condition, such as to exercise a particular program path or to verify compliance with a specific requirement.

So the TEST() macro defines a test case in my opinion, and multiple TEST() together form the test suite, and the documentation might be changed accordingly in Primer.md. Currently it is

TEST() arguments go from general to specific. The first argument is the name of the test case, and the second argument is the test's name within the test case.

In my opinion, the common use of test case and test suite would be reflected by changing this to

TEST() arguments go from general to specific. The first argument is the name of the test suite, and the second argument is the test case's name within the test suite.

Then instead of TEST(test_case_name, test_name) it would be TEST(test_suite_name, test_case_name).

Of course, the code would also need some changes to reflect this.

Maybe this was discussed already and I didn't find the issue. Would be interested to have some discussion on this issue. For my purpose, I just changed the code to output the number of "test suites" instead of "test cases" as shown in pull request #1084 to avoid confusion locally.

[BillyDonahue]

Yes. Thanks for the ISTQB links.
I agree that this is a confusing violation of common conventional terminology.

It goes back to the beginning of gtest and will be very difficult to change, so I think everyone has just gotten used to the backwardsness. What can we really do about it?

[ghost]

Thanks for the reply. Maybe every current user has gotten used to the terminology mismatch, but many potential new users might be confused. I'm currently using Google Test in a study course and to avoid confusing the students, I patched the Google Test sources as in #1084. But of course they might be confused later in case they might use the "original" Google Test once again. It's not a game stopper, but just a little bit annoying.

Why do you think it's very difficult to change? It's "just" wording, and I'd volunteer to propose changes in the documentation.

The harder part might be to convince experienced users in case they rely on the "old" terminology? But even that should'nt be too hard?

[BillyDonahue]

I was referring to the way the terminology is integrated into the public API, mostly as the googletest reflection class "TestCase", part of the UnitTest and TestEventListener APIs. Human-readable documentation and messages should IMO be changed.

[ghost]

I will see if I an do some progress on the documentation.
The APIs could be changed in a later step, if the old API names would be present in parallel to changed API names - would take some time, but should be possible.

[ghost]

I started to look into this. As you noted, changing the API without breaking backward compatibility might be not so easy (though not impossible); changing the documentation is then just some work. And finally even the internal parts of the implementation could be changed, though this is quite some refactoring.

[ghost]

@BillyDonahue - could you please take a look at #1178 ; this might be a start to resolve this nomenclature issue.

[gennadiycivil]

Stay tuned

[gennadiycivil]

This is done now. There is a new API TestSuite and for the time being there is also a deprecated API Case