Running Tests

Why Integration Tests?

The postmark-php project includes a full integration test suite to confirm that all functionality is working before each new release. Given the nature of Postmark, we feel that integration tests are appropriate, as "unit" tests that isolated the library from the Postmark API would not really prove that the client library was compatible/working with the REST API.

Another important point is due to the nature of the integration tests, you may need to run multiple passes to populate some data in your test account. Specifically, some tests rely on stats data to be pre-populated, populating this on every run could make the tests more flakey, as stats are processed asynchronously. Instead, the Postmark team queries against internal test data for these tests, which ensures they run quickly and rarely produce "false negatives."

Setting Testing Keys

To run the tests locally, a copy of the testing_keys.json.example file needs to be copied to testing_keys.json and filled with appropriate values. The testing_keys.json file is ignored from the repo, so it is safe to put private tokens/domains in this file, and be assured that they will not accidentally get published to GitHub.

Using Travis CI with your fork:

If you have a fork of the project set up in Travis CI, you can use set_travis_vars.sh script to publish variables found in testing_keys.json to Travis. Again, these will remain private, as they are encrypted and never displayed publicly by the Travis interface/system.

Downsides of Integration Tests

One of the critical downsides of Integration Tests is that they are inherently harder to configure. As a policy Postmark makes every effort to prevent spam from being sent through our API. One consequence of this policy is that Postmark will not publish any "test" tokens for potential contributors to use for running integration tests of our libraries. The reason for this is that publicly exposing tokens in this repository could allow a spammer to send a limited amount of junk email through our system.

We recognize this policy could penalize potential contributors, but Postmark has been able to achieve a very high inbox rate, and very low time-to-inbox times because we have such a strong policy against junk mail.

Having said all of that, if you'd like to make substantial contributions to the project, please contact Postmark Support, and we will be happy to work with you. For smaller/trivial contributions, feel free to open a Pull Request, and one of the project maintainers will be responsible for ensuring the integration tests have passed before merging it.