Acceptance tests cases for Jenkins and its plug-ins based on selenium and docker.
Switch branches/tags
Clone or download

Acceptance tests for Jenkins

End to end test suite for Jenkins automation server and its plugins.

The scenarios are described ain form of tests controlling Jenkins under test (JUT) through UI / REST API. Clean instance is started for individual tests to isolate the tests. The harness provides convenient docker support so integration tests can be written easily.

Getting Started

The simplest way to start the harness is calling BROWSER=firefox JENKINS_VERSION=2.73 mvn test. Complete test suite takes hours to run due to the number of covered components/use-cases, the cost of Jenkins setup and selenium interactions. That can be avoided by selecting a subset of tests to be run - smoke tests for instance.

Further Reading

Running tests

The harness provides a variety of ways to configure the execution including:

Writing tests

Given how long it takes for the suite to run, test authors are advised to focus on the most popular plugins and use-cases to maximize the value of the test suite. Tests that can or already are written as a part of core/plugin tests should be avoided here as well as tests unlikely to catch future regressions (reproducers for individual bugs, boundary condition testing, etc.). Individual maintainers are expected to update their tests reflecting core/plugin changes as well as ensuring the tests does not produce false positives. Tests identified to violate this guideline might be removed without author's notice for the sake of suite reliability.

Areas where acceptance-tests-harness is more suitable then jenkins-test-harness are:

  • Installing plugins for cross-plugin integration
  • Running tests in realistic classloader environment
  • Verifying UI behaviour in actual web browser