Skip to content

Commit

Permalink
Added "How to test changes in PR" section to documentation
Browse files Browse the repository at this point in the history
Also linked it from PR template.

Closes #936
  • Loading branch information
horenmar authored and philsquared committed Jun 26, 2017
1 parent 7bcfdf8 commit 9382534
Show file tree
Hide file tree
Showing 2 changed files with 24 additions and 2 deletions.
3 changes: 3 additions & 0 deletions .github/pull_request_template.md
Expand Up @@ -2,6 +2,9 @@
Please do not submit pull requests changing the `version.hpp`
or the single-include `catch.hpp` file, these are changed
only when a new release is made.
Before submitting a PR you should probably read the contributor documentation
at docs/contributing.md. It will tell you how to properly test your changes.
-->


Expand Down
23 changes: 21 additions & 2 deletions docs/contributing.md
Expand Up @@ -18,8 +18,8 @@ Ongoing development is currently on _master_. At some point an integration branc
_Users_ of Catch primarily use the single header version. _Maintainers_ should work with the full source (which is still,
primarily, in headers). This can be found in the `include` folder. There are a set of test files, currently under
`projects/SelfTest`. The test app can be built via CMake from the `CMakeLists.txt` file in the root, or you can generate
project files for Visual Studio, XCode, and others (instructions in the `projects` folder). If you have access to CLion
that can work with the CMake file directly.
project files for Visual Studio, XCode, and others (instructions in the `projects` folder). If you have access to CLion,
it can work with the CMake file directly.

As well as the runtime test files you'll also see a `SurrogateCpps` directory under `projects/SelfTest`.
This contains a set of .cpp files that each `#include` a single header.
Expand All @@ -34,6 +34,25 @@ __When submitting a pull request please do not include changes to the single inc
as these are managed by the scripts!__


## Testing your changes

Obviously all changes to Catch's code should be tested. If you added new functionality, you should add tests covering and
showcasing it. Even if you have only made changes to Catch internals (ie you implemented some performance improvements),
you should still test your changes.

This means 3 things

* Compiling Catch's SelfTest project -- code that does not compile is evidently incorrect. Obviously, you are not expected to
have access to all compilers and platforms Catch supports, Catch's CI pipeline will compile your code using supported compilers
once you open a PR.
* Running the SelfTest binary. There should be no unexpected failures on simple run.
* Running Catch's approval tests. Approval tests compare current output of the SelfTest binary in various configurations against
known good output. Catch's repository provides utility scripts `approvalTests.py` to help you with this. It needs to be passed
the SelfTest binary compiled with your changes, like so: `$ ./scripts/approvalTests.py clang-build/SelfTest`. The output should
be fairly self-explanatory.



*this document is still in-progress...*

---
Expand Down

0 comments on commit 9382534

Please sign in to comment.