Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 208 lines (162 sloc) 6.387 kb
08fead5 Added README.TESTING
Yasuo Ohgaki authored
1 [IMPORTANT NOTICE]
2 ------------------
aba1396 Catch up with recent test scripts changes
Yasuo Ohgaki authored
3 Do _not_ ask to developers why some or all tests are failed under
08fead5 Added README.TESTING
Yasuo Ohgaki authored
4 your environment! Let us know if you find why it fails. Thank you.
5
6
aba1396 Catch up with recent test scripts changes
Yasuo Ohgaki authored
7 [Testing Basics]
8 ----------------
9 To execute test scripts, you must build PHP with some SAPI, then you
10 type "make test" to execute all or some test scripts saved under
08fead5 Added README.TESTING
Yasuo Ohgaki authored
11 "tests" directory under source root directory.
12
13 Usage:
14 make test
15
16 "make test" basically executes "run-tests.php" script
17 under source root. Therefore you can execute the script
aba1396 Catch up with recent test scripts changes
Yasuo Ohgaki authored
18 as follows
08fead5 Added README.TESTING
Yasuo Ohgaki authored
19
3de2e94 -c option is for path.
Yasuo Ohgaki authored
20 ./sapi/cli/php -c /path/to/php.ini/ run-tests.php [ext/some_extension_name]
08fead5 Added README.TESTING
Yasuo Ohgaki authored
21
22
23 [Which "php" executable "make test" look for]
24 ---------------------------------------------
96f7bcc Update info and synch example test.
Marcus Boerger authored
25 You must use TEST_PHP_EXECUTABLE environment variable to explicitly
26 select the php executable to be used to run the tests. That can either
27 be the CLI or CGI executable.
28
aba1396 Catch up with recent test scripts changes
Yasuo Ohgaki authored
29 "make test" executes "run-tests.php" script with "php" binary. Some
30 test scripts such as session must be executed by CGI SAPI. Therefore,
31 you must build PHP with CGI SAPI to perform all tests.
08fead5 Added README.TESTING
Yasuo Ohgaki authored
32
aba1396 Catch up with recent test scripts changes
Yasuo Ohgaki authored
33 NOTE: PHP binary executing "run-tests.php" and php binary used for
34 executing test scripts may differ. If you use different PHP binary for
35 executing "run-tests.php" script, you may get errors.
08fead5 Added README.TESTING
Yasuo Ohgaki authored
36
37
38 [Which php.ini is used]
39 -----------------------
40 "make test" force to use php.ini-dist as default config file. If
41 you would like to test with other configuration file, user
42 "run-tests.php" script.
43
44 Example:
96f7bcc Update info and synch example test.
Marcus Boerger authored
45 ./sapi/cli/php -c /path/to/php.ini/ run-tests.php ext/standard
08fead5 Added README.TESTING
Yasuo Ohgaki authored
46
aba1396 Catch up with recent test scripts changes
Yasuo Ohgaki authored
47 If you use php.ini other than php.ini-dist, you may see more failed
48 tests.
08fead5 Added README.TESTING
Yasuo Ohgaki authored
49
50
51 [Which test scripts are executed]
52 ---------------------------------
53 "run-tests.php" ("make test") executes all test scripts by default
54 by looking all directory named "tests". If there are files have "phpt"
55 extension, "run-tests.php" takes test php code from the file and
56 executes it.
57
96f7bcc Update info and synch example test.
Marcus Boerger authored
58 Tester can easily execute tests selectively with as follows.
08fead5 Added README.TESTING
Yasuo Ohgaki authored
59
96f7bcc Update info and synch example test.
Marcus Boerger authored
60 Examples:
3de2e94 -c option is for path.
Yasuo Ohgaki authored
61 ./sapi/cli/php -c /path/to/php.ini/ run-tests.php ext/mbstring
96f7bcc Update info and synch example test.
Marcus Boerger authored
62 ./sapi/cli/php -c /path/to/php.ini/ run-tests.php ext/mbstring/020.phpt
08fead5 Added README.TESTING
Yasuo Ohgaki authored
63
64
65 [Test results]
66 --------------
67 Test results are printed to standard output. If there is a failed test,
68 "run-tests.php" script saves the result, expected result and code
69 executed to the test script directory. For example, if
70 ext/myext/tests/myext.phpt is failed to pass, following files are
71 created:
72
32a77ef Update documentation
Marcus Boerger authored
73 ext/myext/tests/myext.php - actual testfile executed
96f7bcc Update info and synch example test.
Marcus Boerger authored
74 ext/myext/tests/myext.log - log of test execution (L)
75 ext/myext/tests/myext.exp - expected output (E)
76 ext/myext/tests/myext.out - output from test script (O)
77 ext/myext/tests/myext.diff - diff of .out and .exp (D)
08fead5 Added README.TESTING
Yasuo Ohgaki authored
78
96f7bcc Update info and synch example test.
Marcus Boerger authored
79 Tester can verify these files, if failed test is actually a bug or not.
80
81 NOTE: The files generated by tests can be selected by setting the
82 environment variable TEST_PHP_LOG_FORMAT. For each file you want to be
83 generated use the character in brackets as shown above (default is LEOD).
9da3c88 update and correct
Marcus Boerger authored
84 The php file will be generated always.
96f7bcc Update info and synch example test.
Marcus Boerger authored
85
86 NOTE: You can set environment variable TEST_PHP_DETAILED to enable
87 detailed test information.
08fead5 Added README.TESTING
Yasuo Ohgaki authored
88
89
90 [Creating new test files]
91 -------------------------
92 Writing test file is very easy if you are used to PHP. Test file
93 has following format. Here is a actual test file from iconv module.
94
95 ===== ext/iconv/002.phpt =======
96 --TEST--
97 UCS4BE to ASCII
98 --SKIPIF--
99 <?php include('skipif.inc'); ?>
100 --FILE--
101 <?php include('002.inc'); ?>
102 --EXPECT--
103 &#97;&#98;&#99;&#100;
104 abcd
105 ===== end of ext/iconv/002.phpt =======
106
96f7bcc Update info and synch example test.
Marcus Boerger authored
107 As you can see the file has the following sections:
108
9da3c88 update and correct
Marcus Boerger authored
109 "--TEST--" is title of the test.
110 "--SKIPIF--" is condition when to skip this test (optional).
111 "--POST--" is POST variable passed to test script (optional).
112 "--GET--" is GET variable passed to test script (optional).
113 "--INI--" each line contains an ini setting e.g. foo=bar (optional).
114 "--FILE--" is the test script.
115 "--EXPECT--" is the expected output from the test script.
116 "--EXPECTF--" this is the alternate form of --EXPECT--. The difference
117 is is that this form uses sscanf for output validation.
08fead5 Added README.TESTING
Yasuo Ohgaki authored
118
119 ext/iconv/002.phpt uses 2 files. "skipif.inc" is used to skip
120 test when test cannot be performed such as iconv module is not
121 compiled or loaded.
122
123 ==== ext/iconv/skipif.inc ====
124 <?php
125 // This script prints "skip" if condition does not meet.
126
127 if (!extension_loaded("iconv") && ini_get("enable_dl")) {
128 $dlext = (substr(PHP_OS, 0, 3) == "WIN") ? ".dll" : ".so";
129 @dl("iconv$dlext");
130 }
131 if (!extension_loaded("iconv")) {
132 die("skip\n");
133 }
134 ?>
135 ==== end of ext/iconv/skipif.inc ====
136
137 ext/inconv/002.inc is the test script. "run-tests.php" script
138 executes test script with CGI executable.
139
140 ==== ext/iconv/002.inc ===
141 <?php
142 /*
143 Expected output:
144 &#97;&#98;&#99;&#100;
145 abcd
146 */
147
148 $s = unpack("V*", iconv("ascii","UCS-4LE", "abcd"));
149 foreach($s as $c) { print "&#$c;"; } print "\n";
150
151 $s = pack("NNNN", 97, 98, 99, 100);
152 $q = iconv("UCS-4BE", "ascii", $s);
153 print $q; print "\n";
154 ?>
155 === end of ext/iconv/002.inc ===
156
9da3c88 update and correct
Marcus Boerger authored
157 Test script and SKIPIF code should be directly written into *.phpt.
158 However, it is recommended to use include files when more test scripts
159 depend on the same SKIPIF code or when certain test files need the same
160 values for some input.
08fead5 Added README.TESTING
Yasuo Ohgaki authored
161
162
9da3c88 update and correct
Marcus Boerger authored
163 /ext/standard/tests/strings 003.phpt is a good example for using EXPECTF
164 instead of EXPECT. From time to time the algorythmn used for shuffle
165 changed and sometimes the machine used to execute the code has influence
166 on the result of shuffle. But it always retuns a three character string
167 detectable by %s. Other forms scanable are %i for integers, %f for
168 floating point values and %x for hexadecimal values.
169
170 ==== /ext/standard/tests/strings 003.phpt ===
171 --TEST--
172 Testing str_shuffle.
173 --FILE--
174 <?php
175 $s = '123';
176 var_dump(str_shuffle($s));
177 var_dump($s);
178 ?>
179 --EXPECTF--
180 string(3) %s
181 string(3) "123"
182 ==== end of /ext/standard/tests/strings 003.phpt ===
183
08fead5 Added README.TESTING
Yasuo Ohgaki authored
184 [How to help us]
185 ----------------
186 If you find bug in PHP, you can submit bug report AND test script
187 for us. You don't have to write complete script, just give us test
188 script with following format. Please test the script and make sure
189 you write the correct ACTUAL OUTPUT and EXPECTED OUTPUT before you
190 submit.
191
192 <?php
193 /*
194 Bug #12345
195 substr() bug. Do not return expected string.
196
197 ACTUAL OUTPUT
198 XYXA
199
200 EXPECTED OUTPUT
201 ABCD
202 */
203
204 $str = "XYZABCD";
205 echo substr($str,3,7);
206
207 ?>
Something went wrong with that request. Please try again.