Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 151 lines (108 sloc) 5.737 kb
d26f319 Lukas Larsson Add a testing HOWTO
garazdawi authored
1 Testing Erlang/OTP
2 ==================
3
4 Before you start testing you need to have the Erlang release which you
5 are going to test in your path. See [$ERL_TOP/HOWTO/INSTALL.md][] for
6 instructions on how to build an Erlang release.
7
8 Short version
9 -------------
10 Move to the top directory of the Erlang release you want to test, i.e.
11 cd /ldisk/work/otp
12
13 export ERL_TOP=`pwd`
14 ./otp_build setup -a
15 export PATH=`pwd`/bin:$PATH
16 ./otp_build tests
17 cd release/tests/test_server
18 erl -s ts install -s ts run all_tests -s init stop
19
20 Where are the tests
21 -------------------
22
23 There are a lot of tests which test Erlang/OTP (as of 2012 about 12000) and
24 they are located in different places throughout the source tree. Below is a list
25 of the places where you can expect to find tests:
26
27 * $ERL_TOP/lib/AppName/test/
28 * $ERL_TOP/erts/test/
29 * $ERL_TOP/erts/emulator/test/
30 * $ERL_TOP/erts/epmd/test/
31
32 Writing tests
33 -------------
34
35 All tests are [common_test][] suites and follow the same pattern as all
36 [common_test][] suites. However, a couple of corner cases are
37 handled by a test wrapper called `ts`. `ts` allows the test writer to put
38 `Makefile.src` and `Makefile.first` files in the [data_dir][] and a special
39 directory called `all_SUITE_data`.
40
41 `Makefile.first` is run before any other Makefile and is typically used to
42 generate .hrl files which are needed by other test suites. At the moment only
43 the erl_interface tests use this feature and it should remain that way.
44
45 `Makefile.src` is configured to a `Makefile` using the `variables` created when
46 [configuring the tests][]. These `Makefile`s are later run by the test suite
47 to compile whatever platform specific code the tests need to run.
48
49 Releasing tests
50 ---------------
51
52 If you cannot use [ct_run][] in the source tree you have to release the tests
53 into a common test directory. The easiest way to do this is to use `otp_build`
54 like this:
55
56 export ERL_TOP=`pwd`; ./otp_build tests
57
58 This will release all tests in Erlang/OTP to `$ERL_TOP/release/tests/`. If you
59 want to change the directory where the tests are released to use the `TESTROOT`
60 environmental variable.
61
62 In the `$TESTROOT` you should now see *_test folders. These folders contain
63 everything needed to test Erlang/OTP and are platform independent; if you are
64 testing Erlang on multiple platforms you just have to release on one and copy
65 the tests to all other platforms.
66
67 ### Releasing cross tests
68
69 For releasing tests in a cross compilation environment see [$ERL_TOP/HOWTO/INSTALL-CROSS.md][].
70
71 Configuring and Running tests
72 -----------------------------
73
74 Running tests is done by first navigating to the `$TESTROOT/test_server` folder
75 created when you released the tests and then start `erl` in that directory. The
76 emulator flags specified will be used in the test runs. For example, if you want
77 to test using async threads you have to supply `+A 10` to `erl` when you start it.
78
79 To configure and run the tests `ts` is used. `ts` is a wrapper module to
80 [common_test][] which takes care of configuration and build issues before
81 [common_test][] is started.
82
83 `ts` has a lot of special options and functions which can be usefull when
84 testing Erlang/OTP. For a full listing issue `ts:help()` in the erlang shell.
85
86 ### Configuring the test environment
87
88 Before running released tests you have to install them on the target system.
89 Installing the tests is done by calling `ts:install().` in the Erlang shell
90 which you intend to test from. `ts:install()` is basically a wrapper to a
91 configure script and some Erlang code which figures out what your system looks
92 like and what kind of emulator you are testing with. `ts:install()` can also
93 take some arguments if necessary, see `ts:help()` for details.
94
95 All variables created by `ts:install()` are found in
96 `$TESTROOT/test_server/variables`.
97
98 ### Running the tests
99
100 To run all test suites go to `$TESTROOT/test_server` fire up an Erlang shell and type:
101
102 ts:run().
103
104 Note that running all tests will require several hours, so you may want to run
105 the test cases for a single application
106
107 ts:run(Application, [batch]).
108
109 or even part of the test suite for an application, for example
110
111 ts:run(emulator, bs, [batch]).
112
113 to run all test suite modules starting with `bs` (i.e. all modules that test
114 the bit syntax).
115
116 To run a specific test case in a module, the full name of the module and test
117 case must be specified:
118
119 ts:run(emulator, bs_bincomp_SUITE, byte_aligned, [batch]).
120
121 Run `ts:help().` for more information.
122
123 As of R14B02 it is also possibly to start all tests but the erl_interface tests
124 by invoking Common Test directly from the released applications test directory,
125 i.e.
126
127 cd $TESTROOT/test_server
128 $ERL_TOP/bin/ct_run -suite ../compiler_test/andor_SUITE -case t_orelse
129
130 Running [ct_run][] from the command line still requires you to do the
131 `ts:install()` step above.
132
133 Examining the results
134 ---------------------
135
136 Open the file `release/tests/test_server/index.html` in a web browser. Or open
137 `release/tests/test_server/last_test.html` when a test suite is running to
138 examine the results so far for the currently executing test suite (in R14B02 and
139 later you want to open the `release/tests/test_server/all_runs.html` file to
140 get to the currently running test)
141
142 [ct_run]: http://www.erlang.org/doc/man/ct_run.html
143 [ct hook]: http://www.erlang.org/doc/apps/common_test/ct_hooks_chapter.html
144 [$ERL_TOP/HOWTO/INSTALL.md]: INSTALL.md
145 [$ERL_TOP/HOWTO/INSTALL-CROSS.md]: INSTALL-CROSS.md#testing-the-cross-compiled-system
146 [common_test]: http://www.erlang.org/doc/man/ct.html
147 [data_dir]: http://www.erlang.org/doc/apps/common_test/write_test_chapter.html#data_priv_dir
148 [configuring the tests]: #configuring-the-test-environment
149
150 [?TOC]: true
Something went wrong with that request. Please try again.