Skip to content

Test Suite Configuration Manifesto

Jump to bottom Edit New page
Robert Schultheis edited this page Mar 22, 2014 · 3 revisions

As the creators, maintainers, and users of automated test suites we have encountered a diverse collection of ways to control a test suite. The time has come to agree as a community on a common approach that will work for most standalone automated test suites. The common approach shall be backed by pragmatic, common-sense, and simple principle. With that in mind...

We hereby declare that executors of test suites have the following rights:

  • You shall be able to control your test suite using environment variables. You shall not be required to edit config files in order to run your test suite in the desired way. This decree will ensure your test suite integrates easily into any continuous integration system.

  • Your test configuration shall make it easy to support test execution against your various environments (QA, Staging, Production, local development, etc). You shall be able to control the environment using a single environment variable.

Accordingly, we enact the following principles that shall govern the land of test configuration:

  • Configuration should be used only when needed. The convention-over-configuration paradigm should be used wherever possible to avoid excessive configuration.

  • Test suite configuration should generally not be conflated with test data or reference data. Separate management of test data is encouraged except for cases where only a very small amount of test data is needed.

Add a custom footer
Add a custom sidebar
Clone this wiki locally