Skip to content

Commit

Permalink
docs: document some testing patterns
Browse files Browse the repository at this point in the history
This change adds documentation of:
* The TestApi pattern
* The ForTesting method pattern
* The Friend-the-tests pattern

The TestApi doc references another doc, describing the Passkey pattern; that doc
is included in another CL but not yet committed:
<https://chromium-review.googlesource.com/c/chromium/src/+/1739552>

Bug: None
Change-Id: Idf6bd9385e1912e3653894fd89381751cc2cb115
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1746647
Reviewed-by: Nico Weber <thakis@chromium.org>
Commit-Queue: Elly Fong-Jones <ellyjones@chromium.org>
Cr-Commit-Position: refs/heads/master@{#686043}
  • Loading branch information
Elly Fong-Jones authored and Commit Bot committed Aug 12, 2019
1 parent 4f753a6 commit c965059
Show file tree
Hide file tree
Showing 3 changed files with 177 additions and 0 deletions.
52 changes: 52 additions & 0 deletions docs/patterns/fortesting-methods.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# The ForTesting Methods Pattern

The ForTesting pattern involves creating public helper methods on a class to
provide access to test-only functionality, and giving them a specific type of
name (`XForTesting` or `XForTest`) to provide a signal at the call site that they
are not intended for regular use.

## Use this pattern when:

You have a widely-used object that you need to expose a small amount of test
functionality on.

## Don't use this pattern when:

* You have lots of different ForTesting methods: consider the [TestApi] pattern
instead.
* Only a small set of test cases need access: consider the [friend the tests]
pattern instead, to avoid polluting the public API.

## Alternatives / See also:

* [Friend the tests]
* [TestApi]
* [ForTesting in the style guide](../../styleguide/c++/c++.md)

## How to use this pattern:

```
class Foo {
public:
// ... regular public API ...
void DoStuffForTesting();
};
```

The ForTesting suffix indicates to code reviewers that the method should not be
called in production code. There is a very similar antipattern in which the
suffix is missing:

```
class Foo {
public:
// ... regular public API ...
// Do not call! Only for testing!
void DoStuff();
};
```

[testapi]: testapi.md
[friend the tests]: friend-the-tests.md
52 changes: 52 additions & 0 deletions docs/patterns/friend-the-tests.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# The Friend-the-tests Pattern

The Friend-the-tests pattern involves friending test fixture classes from the
class under test. Chromium has a macro named `FRIEND_TEST_ALL_PREFIXES` that
makes this convenient to do.

**Note**: Friending test classes is usually not the best way to provide testing
access to a class. It makes it far too easy to unintentionally depend on the
class's implementation rather than its interface. It is better to use a proper
testing interface than to friend test classes.

## Use this pattern when:

A test needs simple access to the internals of a class, such as access to
individual fields or methods.

## Don't use this pattern when:

* Many different test classes need access to a class's internals: you will have
to friend too many things and the class under test will end up coupled with
too many unit tests.
* You're doing it to test private methods: in general, you should not separately
test private methods; the correctness of the private methods should be
observable from the correctness of the public methods, and if a piece of
private behavior has no effect on the public behavior of a class, it might
actually be unnecessary.

## Alternatives / See also:

* [TestApi]
* [ForTesting methods]
* ["Test the contract, not the implementation"](https://abseil.io/tips/135)
* [Discussion thread on cxx@](https://groups.google.com/a/chromium.org/d/msg/cxx/AOsKCPCBYhk/RDVLSMRGCgAJ)

## How to use this pattern:

```
class Foo {
public:
// ... public API ...
private:
FRIEND_TEST_ALL_PREFIXES(FooTest, TestSomeProperty);
FRIEND_TEST_ALL_PREFIXES(FooTest, TestABehavior);
// or if every test in the suite needs private access:
friend class FooTest;
};
```

[fortesting methods]: fortesting-methods.md
[testapi]: testapi.md
73 changes: 73 additions & 0 deletions docs/patterns/testapi.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# The TestApi Pattern

The TestApi pattern involves creating a helper class to provide access to
test-only functionality on a commonly-used class.

## Use this pattern when:

You have a widely-used object that you need to expose test functionality on, but
you want to be confident that this test functionality is not used outside tests.

## Don't use this pattern when:

* The commonly-used class only needs "test access" from its own tests or
closely-related tests; in this case, simply [friend the tests].
* Only a handful of simple test-access methods are needed, like trivial setters;
in this case, the [ForTesting methods] pattern is lighter-weight.

## Alternatives / See also:

* [Friend the tests]
* [ForTesting methods]

## How to use this pattern:

`//foo/commonly_used.h`:
```
class CommonlyUsed {
public:
// ... big public API ...
private:
friend class CommonlyUsedTestApi;
// Any private methods to be used by CommonlyUsedTestApi, possibly using the
// [passkey pattern](passkey.md).
};
```

`//foo/commonly_used_test_api.h`:
```
class CommonlyUsedTestApi {
public:
CommonlyUsedTestApi(CommonlyUsed* thing);
void DoTestStuff(...);
// or maybe just:
static void DoTestStuff(CommonlyUsed* thing, ...);
// and maybe also:
std::unique_ptr<CommonlyUsed> CreateTestCommonlyUsed(...);
};
```

And then client code can do:
```
CommonlyUsedTestApi(commonly_used).DoTestStuff(...);
```

Then only link `commonly_used_test_api.{cc,h}` in test targets, so these methods
cannot accidentally be used in production code. This way, CommonlyUsed is not
polluted with code paths that are only used in tests, and code that has "test
access" to CommonlyUsed is very easy to find. Also, coupling between
CommonlyUsed and tests that exercise its test functionality is reduced.

Note that this pattern should be used only judiciously - an extensive TestApi is
often a sign that a class is doing too much or has too much internal state, or
is simply un-ergonomic to use and therefore requires extensive setup. Also, if
many different tests need access to test-only functionality, that may indicate a
gap in the class under test's public API.

[friend the tests]: friend-the-tests.md
[fortesting methods]: fortesting-methods.md

0 comments on commit c965059

Please sign in to comment.