Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 195 lines (136 sloc) 8.185 kb
03e60ee7 »
2012-04-30 Initial commit of working example.
1 # Proof of Concept: Running Tests from thirdparty-jar
2
3 ## The Story
4
5 **In order to** run a given set of tests in different workspaces and/or environments
6 **As** a tester
7 **I want to** execute deployed tests from the repository bundled as JAR.
8
9 ## Discussion
10
11 Some of you might know this use case:
12
13 * You have one product which combines many submodules.
14
15 * All submodules are developed by independent development teams.
16
17 * Each submodule has its own integration tests.
18
19 * What is missing is a system test for all the submodules combined in the product.
20
21 * What you wish to do is to run tests from the submodules again on the product.
22
23 ## The Problem
24
25 Exactly this use case can be found in the [Maven Failsafe Plugin FAQ][]:
26 [How can I reuse my test code in other modules?][FAQ-reuse-test-code]
27
28 The FAQ points to the [Maven Guide to using attached tests][maven-guide-attached-tests] (which again points to the
29 [test-jar-Mojo][]). Quite helpful, as you learn that there is a packaging-type `test-jar` which you might use for this
30 purpose.
31
32 Unfortunately there is one problem: While from the artifact side everything is well prepared the main Plugins to
33 run the tests don't support this feature, which are:
34
35 * [Maven Failsafe Plugin][]
36 * [Maven Surefire Plugin][]
37
38 This issue is reported as [SUREFIRE-569][] and is unresolved since 2009. So it is unlikely that it will ever be fixed.
39
40 ## The Solutions
41
42 ### Extract Using Dependency Plugin
43
44 A suggested solution in the comments to [SUREFIRE-569][] is to use the dependency plugin and extract the test sources
2e725e9d »
2012-04-30 Fix typos.
45 to the path `target/test-classes`.
03e60ee7 »
2012-04-30 Initial commit of working example.
46
47 **Drawback:** You have to fiddle around with the build process.
48
49 ### Use Suite Files
50
51 An alternative is to agree on a kind of Test-API between the main project and its submodules. Every submodule could
52 deliver a suite file like for example:
53
54 ```java
55 @RunWith(Suite.class)
56 @Suite.SuiteClasses({
57 Module1FirstITest.class,
58 Module1SecondITest.class
59 })
60 public class Module1ITestSuite {
61 }
62 ```
63
64 Now the main project can use these suite files (packaged with packaging type `test-jar`) to run the tests as suite:
65
66 ```java
67 @RunWith(Suite.class)
68 @Suite.SuiteClasses({Module1ITestSuite.class, Module2ITestSuite.class})
69 public class SystemITestSuite {
70 }
71 ```
72
73 **Drawback:**
74
75 * You will need to manually add test classes as suite.
76
77 **Advantages:**
78
79 * Your submodule projects can easily control which tests will be executed in the system test.
80
81 * The system test can exclude tests from submodules (if they are not ready yet for example). As further enhancement you
82 might decide to have such a hook-class for every submodule so that you can even exclude them at command line.
83
84 * The reporting looks nice. For example in your IDE you will have a result tree like this:
85
86 > * `SystemITestSuite`
87 > * `Model1ITestSuite`
88 > * `Module1FirstITest`
89 > * `shouldWorkThat`
90 > * `shouldWorkThis`
91 > * `Model2ITestSuite`
92
93 ## Advanced Use Case
94
95 If this is really an advanced use case is hard to answer. I guess this is the default use case when having system
96 tests. You most likely will have a server to test. And this server has a host and port to connect to. While this
97 is a local test server for the submodule the whole product might run on a totally different server.
98
99 So the question is: *How to run the same tests with a different environment?*
100
101 The easiest answer on this I found is using [Spring][]. You can provide configurations as context-configurations and
102 access them in your tests via Injection. I have two solutions in mind which I will mention here:
103
104 ### Environment Class with @Value-annotations
105
106 First of all you might use an environment class as managed bean which holds properties like this:
107
108 ```java
109 @Value("${server.host:localhost}")
110 private String host;
111 ```
112
113 Your tests will use this configuration to get all environmental parameters.
114
115 **Drawback:** The properties which you might set from the outside are hidden in this class. You have to document them
116 clearly and be careful on refactorings. Actually you will never be able to rename the properties. And for the product
117 tests it requires that all properties from the different modules come with a clear naming scheme to avoid collisions.
118
119 Having this you might end up with properties like `module1.server.host` and `module2.server.host` where actually both
120 are the same. Or even worse with `module1.server.host` and `module1.server.port` and in contrast wit
121 `module2.server.url` which is actually the same as `http://${module1.server.host}:${module1.server.port}/`.
122
123 ### Environment Class instantiated in Context Configuration
124
125 To solve the previous drawback is to leave it up to the tests to configure the properties. Environment classes are
126 instantiated through the Context configuration.
127
128 The API you define is the location of the context configuration and the class which contains the configuration. Your
129 submodule tests then will need a base class configured like this:
130
131 ```java
132 @RunWith(SpringJUnit4ClassRunner.class)
133 @ContextConfiguration(locations={"classpath:/META-INF/testartifacts/myproject-itest-context.xml"})
134 public abstract class MyProjectBaseITestCase {
135 }
136 ```
137
138 And in the folder `src/test/resources/META-INF/testartifacts` of your submodule's integration tests you add a file
139 `myproject-itest-context.xml`.
140
141 Now as being the product who wants to run the tests all you have to do is override `myproject-itest-context.xml`.
142
143 The XML-File itself might look like this:
144
145 ```xml
146 <?xml version="1.0" encoding="UTF-8"?>
147 <beans xmlns="http://www.springframework.org/schema/beans"
148 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
149 xmlns:context="http://www.springframework.org/schema/context"
150 xsi:schemaLocation="http://www.springframework.org/schema/beans
151 http://www.springframework.org/schema/beans/spring-beans.xsd
152 http://www.springframework.org/schema/context
153 http://www.springframework.org/schema/context/spring-context.xsd">
154
155 <!-- Override through system properties; user defaults in USER_HOME/.module.test.properties -->
156
157 <context:property-placeholder
158 ignore-resource-not-found="true"
159 ignore-unresolvable="true"
160 local-override="true"
161 system-properties-mode="OVERRIDE"
162 order="1"
163 location="file:${user.home}/.module.test.properties"/>
164
165 <context:component-scan base-package="poc.runtest.testartifacts.itests"/>
166
167 <bean id="myproject_environment" class="poc.runtest.testartifacts.itests.MyProjectEnvironment">
168 <property name="serverName" value="Project: ${server.name:NameFromProjectXml}"/>
169 <property name="serverUrl" value="http://${server.host:HostFromProjectXml}:${server.port:PortFromProjectXml}/"/>
170 </bean>
171
172 </beans>
173 ```
174
175 And the environment-snippet for the product might look like this:
176
177 ```xml
178 <bean id="myproject_environment" class="poc.runtest.testartifacts.itests.MyProjectEnvironment">
179 <property name="serverName" value="System: ${server.name:NameFromSystemXml}"/>
180 <property name="serverUrl" value="http://${server.host:HostFromSystemXml}:${server.port:PortFromSystemXml}/"/>
181 </bean>
182 ```
183
184 ## Test It
185
186 Run `mvn clean verify` to see the system tests running by the failsafe-plugin.
187
188 [Maven Failsafe Plugin]: <http://maven.apache.org/plugins/maven-failsafe-plugin/> "Maven Failsafe Plugin"
189 [Maven Surefire Plugin]: <http://maven.apache.org/plugins/maven-surefire-plugin/index.html> "Maven Surefire Plugin"
190 [Maven Failsafe Plugin FAQ]: <http://maven.apache.org/plugins/maven-failsafe-plugin/faq.html> "maven-failsafe-plugin: Frequently Asked Questions"
191 [FAQ-reuse-test-code]: <http://maven.apache.org/plugins/maven-failsafe-plugin/faq.html#reuse-test-code> "maven-failsafe-plugin: Frequently Asked Questions"
192 [test-jar-Mojo]: <http://maven.apache.org/plugins/maven-jar-plugin/test-jar-mojo.html> "maven-jar-plugin: test-jar"
193 [maven-guide-attached-tests]: <http://maven.apache.org/guides/mini/guide-attached-tests.html> "Maven Guide for executing attached tests"
194 [SUREFIRE-569]: <http://jira.codehaus.org/browse/SUREFIRE-569> "[SUREFIRE-569] There should be a way to run unit tests from a dependency jar"
195 [Spring]: <http://www.springsource.org/> "SpringSource.org"
Something went wrong with that request. Please try again.