Before you start testing you need to have the Erlang release which you are going to test in your path. See $ERL_TOP/HOWTO/INSTALL.md for instructions on how to build an Erlang release.
Move to the top directory of the Erlang release you want to test, i.e. cd /ldisk/work/otp
export ERL_TOP=`pwd` ./otp_build setup -a export PATH=`pwd`/bin:$PATH ./otp_build tests cd release/tests/test_server erl -s ts install -s ts run all_tests -s init stop
Where are the tests
There are a lot of tests which test Erlang/OTP (as of 2012 about 12000) and they are located in different places throughout the source tree. Below is a list of the places where you can expect to find tests:
All tests are common_test suites and follow the same pattern as all
common_test suites. However, a couple of corner cases are
handled by a test wrapper called
ts allows the test writer to put
Makefile.first files in the data_dir and a special
Makefile.first is run before any other Makefile and is typically used to
generate .hrl files which are needed by other test suites. At the moment only
the erl_interface tests use this feature and it should remain that way.
Makefile.src is configured to a
Makefile using the
variables created when
configuring the tests. These
Makefiles are later run by the test suite
to compile whatever platform specific code the tests need to run.
If you cannot use ct_run in the source tree you have to release the tests
into a common test directory. The easiest way to do this is to use
export ERL_TOP=`pwd`; ./otp_build tests
This will release all tests in Erlang/OTP to
$ERL_TOP/release/tests/. If you
want to change the directory where the tests are released to use the
$TESTROOT you should now see *_test folders. These folders contain
everything needed to test Erlang/OTP and are platform independent; if you are
testing Erlang on multiple platforms you just have to release on one and copy
the tests to all other platforms.
Releasing cross tests
For releasing tests in a cross compilation environment see $ERL_TOP/HOWTO/INSTALL-CROSS.md.
Configuring and Running tests
Running tests is done by first navigating to the
created when you released the tests and then start
erl in that directory. The
emulator flags specified will be used in the test runs. For example, if you want
to test using async threads you have to supply
+A 10 to
erl when you start it.
ts has a lot of special options and functions which can be usefull when
testing Erlang/OTP. For a full listing issue
ts:help() in the erlang shell.
Configuring the test environment
Before running released tests you have to install them on the target system.
Installing the tests is done by calling
ts:install(). in the Erlang shell
which you intend to test from.
ts:install() is basically a wrapper to a
configure script and some Erlang code which figures out what your system looks
like and what kind of emulator you are testing with.
ts:install() can also
take some arguments if necessary, see
ts:help() for details.
All variables created by
ts:install() are found in
Running the tests
To run all test suites go to
$TESTROOT/test_server fire up an Erlang shell and type:
Note that running all tests will require several hours, so you may want to run the test cases for a single application
or even part of the test suite for an application, for example
ts:run(emulator, bs, [batch]).
to run all test suite modules starting with
bs (i.e. all modules that test
the bit syntax).
To run a specific test case in a module, the full name of the module and test case must be specified:
ts:run(emulator, bs_bincomp_SUITE, byte_aligned, [batch]).
ts:help(). for more information.
As of R14B02 it is also possibly to start all tests but the erl_interface tests by invoking Common Test directly from the released applications test directory, i.e.
cd $TESTROOT/test_server $ERL_TOP/bin/ct_run -suite ../compiler_test/andor_SUITE -case t_orelse
Running ct_run from the command line still requires you to do the
ts:install() step above.
Examining the results
Open the file
release/tests/test_server/index.html in a web browser. Or open
release/tests/test_server/last_test.html when a test suite is running to
examine the results so far for the currently executing test suite (in R14B02 and
later you want to open the
release/tests/test_server/all_runs.html file to
get to the currently running test)