Permalink
Browse files

Update README.txt

  • Loading branch information...
OndraZizka committed Oct 23, 2011
1 parent 8bde17e commit 2abab50de3c2a16c5e17151fee2077141f9af32c
Showing with 44 additions and 29 deletions.
  1. +44 −29 testsuite/README.txt
View
@@ -5,73 +5,82 @@ This directory contains the AS7 testsuite in its new format, as described in htt
Some points to note:
+
1. Defined modules
+==================
The testsuite has the following maven modules:
-* api - J2EE API tests
-* benchmark - benchmarking integration tests
-* domain - domain management integration tests (require special framework)
+* benchmark - benchmarking integration tests
+* compat - compatibility tests (require special dependencies)
+* domain - domain management integration tests (require special framework)
* integration - general integration tests
-* spec - JBoss specific API tests
-* stress - stress integration tests
+* stress - stress integration tests
+
2. Defined directories and files
+================================
+
+ testsuite/pom.xml - this has a header describing the purpose of the pom and what should and should not go in it
+ testsuite/integration/pom.xml - this has a header describing the purpose of the pom and what should and should not go in it
-testsuite/pom.xml - this has a header describing the purpose of the pom and what should and should not go in it
-testsuite/integration/pom.xml - this has a header describing the purpose of the pom and what should and should not go in it
+For each logical grouping of tests X (e.g. basic integration, clustering, compat, smoke, ...), we use the following locations:
-For each logical grouping of tests X (e.g. basic integration, clustering, compat, smoke, timerservice,...), we use
-the following locations:
+ src/test/java/org/jboss/as/X - holds all sources and test-specific resources for the X tests
+ src/test/resources/X - holds resources shared by X tests, which include resources for creating deployments
+ as well as Arquillian configuration files
-src/test/java/org/jboss/as/X - holds all sources for the X tests
-src/test/resources/clustering - holds all resources for the X tests, which include resources for creating deployments
- as well as Arquillian configuration files
+ src/test/resources/test-configs/ - holds all files to be copied into custom configurations
+ src/test/scripts - holds ant build scripts for tasks which require execution of maven-antrun-plugin (e.g building configs)
+ src/test/xslt - holds XSLT stylesheets for making changes to AS XML-based configuration files
-src/test/resources/test-configs/ - holds all files to be copied into custom configurations
-src/test/scripts - holds ant build scripts for tasks which require execution of maven-antrun-plugin (e.g building configs)
-src/test/xslt - holds XSLT stylesheets for making changes to AS XML-based configuration files
3. Defined profiles
+===================
maven build profiles are used to control:
* testsuite modules enabled
* test cases to be executed within a module
3.1 testsuite modules
+---------------------
+
- for each module X, the profile is named X.module.profile and is enabled by -DX.module
- for example, integration.module.profile, enabled by -Dintegration.module, enables the integration module
- there are profiles for api, benchmark, domain, integration, spec and stress
3.2 test cases to be executed
+---------------------
+
- for each logical set of test cases X in module Y, the profile is named X.Y.tests.profile and is enabled by -DX.Y.tests
-- for example, the clustering tests are defined in profile clustering.integration.tests.profile, enabled by
--Dclustering.integration.tests
+- e.g., the clustering tests are defined in profile clustering.integration.tests.profile, enabled by -Dclustering.integration.tests
- in the integration module, there are profiles for the following logical groups:
-- basic.integration.tests.profile
-- clustering.integration.tests.profile
-- smoke.integration.tests.profile
--- timerservice.integration.tests.profile
--- compat.integration.tests.profile
As time goes on, more logical groupings of tests into profiles will emerge. These are required when a set of
tests needs to execute with custom server builds/configurations, server startup parameters or client side parameters.
All of these are activated independently of each other, unlike the -P usage.
+
4. Executing tests from the command line
+========================================
-Tests are run using the build.sh and build.bat shell scripts in the main build directory.
-At present, running the integration tests also runs the unit tests.
+Tests are run either when running the build.sh and build.bat shell scripts in the main build directory,
+or using integration-tests.sh / .bat. The first way also runs the unit tests.
# run the default module and its tests (-Dintegration.module -Dsmoke.integration.tests)
> ./build.sh clean test
# run a selected set of modules and tests
-> ./build.sh clean test -Dspec.module -Ddomain.module -Dintegration.module -Dbasic.integration.tests
+> ./build.sh clean test -Ddomain-tests -Dintegration.module -Dbasic.integration.tests
+
5. Building server configurations
+=================================
Whenever we fire a surefire execution, this causes Arquillian to start a server(s), execute tests and then shutdown
the server(s). Starting a server in multiple executions kills the logs of previous executions, and so it's recommended
@@ -85,6 +94,7 @@ When we build a server configuration, we need to carry out several steps:
Building server configurations is achieved in two different ways, depending on requirements:
5.1 maven-resources-plugin based build
+--------------------------------------
In the shared build of target/jbossas in testsuite/pom.xml, i'm creating the server configuration using
the maven-resources-plugin, but this requires two separate executions:
@@ -95,31 +105,34 @@ the maven-resources-plugin, but this requires two separate executions:
This is effectively the way buildmagic used to build configurations. We have to include all of this stuff for every
configuration we build, which is a lot of lines of plugin configuration.
-Parametrization is achieved by XSLT transforms (changeIPAddresses, changeDatabase, addPortOffset)
+Parametrization is achieved by XSLT transforms (changeIPAddresses.xsl, changeDatabase.xsl, addPortOffset.xsl).
5.2 maven-ant-plugin based build
+--------------------------------
For almost all other server configurations, maven-antrun-plugin is used, which is much more compact.
A single ant target is called to construct the build of the server, using one or more of:
- copying jars from target/jdbcDrivers
- using test-configs and ant filtering to copy in updated files into the new configuration
-- using XSLT stylesheets to automatically perform common editing tasks on the server's XML configuration files (e.g. IP
-addresses changed with setIPAddresses.xsl, database subsystems configuration changed with changeDefaultdatabase.xsl, etc)
+- using XSLT stylesheets to automatically perform common editing tasks on the server's XML configuration files
+ (e.g. IP addresses changed with setIPAddresses.xsl, database conf. changed with changeDefaultdatabase.xsl, etc.)
5.3 Sharing build configurations
+--------------------------------
The testsuite pom in testsuite/pom.xml contains a surefire execution for building a server
configuration under target/jboss-as. This is used as a default version of the configuration which is inherited by
all modules and is used by tests not requiring a specially modified server configuration.
Corresponds to the all config in buildmagic.
-Configurations for tests which do require their own configuration are built individually in the profile
-in question. For example, clustering.integration.tests.profile contains executions to build clustering-udp-0,
-clustering-udp-1.
+Configurations for tests which do require their own configuration are built individually in the profile in question.
+For example, clustering.integration.tests.profile contains executions to build clustering-udp-0, clustering-udp-1.
+
6. Parametrization of testsuite runs
+------------------------------------
The testsuite/pom.xml contains a properties section which should hold all globally defined properties required to run
the testsuite and their defaults (e.g. IP addresses, IP stack configs, database configs, etc).
@@ -130,12 +143,14 @@ on the command line, such as:
> mvn clean test -Dintegration.module -Dclustering.integration.tests -Dnode0=192.168.0.1 -Dnode1=192.168.0.2
+
7. Changing the database
+------------------------
The prototype also contains a profile which shows how the default database for AS 7 can be changed.
> mvn clean test -Dds=mysql51 <module targets> <test targets>
- will run the tests using MySQL version 5.1 instead of the default H2 in-memory database.
+will run the tests using MySQL 5.1 instead of the default H2 database.

0 comments on commit 2abab50

Please sign in to comment.