Skip to content


Subversion checkout URL

You can clone with
Download ZIP
Browse files

Some english fixes

Added docs for automated testing.
Now telling the current version of the truth about ini settings :)
Be more verbose about QA goals intentions and the tester's part in the
  • Loading branch information...
commit 8bfb6c58930ce333a5df6735132285e1e4979ac2 1 parent 1de45a0
Melvyn Sopacua authored
Showing with 46 additions and 23 deletions.
  1. +46 −23 README.TESTING
@@ -14,10 +14,11 @@ Usage:
make test
"make test" basically executes "run-tests.php" script
-under source root. Therefore you can execute the script
-as follows
+under the source root (parallel builds will not work). Therefore you
+can execute the script as follows:
-./sapi/cli/php -c /path/to/php.ini/ run-tests.php [ext/some_extension_name]
+TEST_PHP_EXECUTABLE=sapi/cli/php \
+sapi/cli/php [-c /path/to/php.ini] run-tests.php [ext/foo/tests/GLOB]
[Which "php" executable "make test" look for]
@@ -37,38 +38,39 @@ executing "run-tests.php" script, you may get errors.
[Which php.ini is used]
- "make test" force to use php.ini-dist as default config file. If
-you would like to test with other configuration file, user
-"run-tests.php" script.
-./sapi/cli/php -c /path/to/php.ini/ run-tests.php ext/standard
-If you use php.ini other than php.ini-dist, you may see more failed
+ "make test" uses the same php.ini file as it would once installed.
+The tests have been written to be independant of that php.ini file,
+so if you find a test that is affected by a setting, please report
+this, so we can address the issue.
[Which test scripts are executed]
- "run-tests.php" ("make test") executes all test scripts by default
-by looking all directory named "tests". If there are files have "phpt"
-extension, "run-tests.php" takes test php code from the file and
-executes it.
+ "run-tests.php" ("make test"), without any arguments executes all
+test scripts by looking extracting all directories named "tests"
+from the source root and any subdirectories below. If there are files,
+which have a "phpt" extension, "run-tests.php" looks at the sections
+in these files, determines whether it should run it, by evaluating
+the 'SKIP' section. If the test is elligable for execution, the 'FILE'
+section is extracted into a ".php" file, with a unique name and
+When an argument is given or TESTS environment variable is set, the
+GLOB is expanded by the shell and any file with extension "*.phpt" is
+regarded as a testfile.
Tester can easily execute tests selectively with as follows.
-./sapi/cli/php -c /path/to/php.ini/ run-tests.php ext/mbstring
-./sapi/cli/php -c /path/to/php.ini/ run-tests.php ext/mbstring/020.phpt
+./sapi/cli/php run-tests.php ext/mbstring/*
+./sapi/cli/php run-tests.php ext/mbstring/020.phpt
[Test results]
Test results are printed to standard output. If there is a failed test,
-"run-tests.php" script saves the result, expected result and code
-executed to the test script directory. For example, if
-ext/myext/tests/myext.phpt is failed to pass, following files are
+the "run-tests.php" script saves the result, the expected result and the
+code executed to the test script directory. For example, if
+ext/myext/tests/myext.phpt fails to pass, the following files are created:
ext/myext/tests/myext.php - actual testfile executed
ext/myext/tests/myext.log - log of test execution (L)
@@ -76,7 +78,13 @@ ext/myext/tests/myext.exp - expected output (E)
ext/myext/tests/myext.out - output from test script (O)
ext/myext/tests/myext.diff - diff of .out and .exp (D)
- Tester can verify these files, if failed test is actually a bug or not.
+ Failed tests are always bugs. Either the test is bugged or not considering
+factors applying to the tester's environment, or there is a bug in PHP.
+If this is a known bug, we strive to provide bugnumbers, in either the
+test name or the file name. You can check the status of such a bug, by
+going to: where 12345 is the bugnumber.
+For clarity and automated processing, bugnumbers are prefixed by a hash
+sign '#' in testnames and/or testcases are named bug12345.phpt.
NOTE: The files generated by tests can be selected by setting the
environment variable TEST_PHP_LOG_FORMAT. For each file you want to be
@@ -86,6 +94,21 @@ The php file will be generated always.
NOTE: You can set environment variable TEST_PHP_DETAILED to enable
detailed test information.
+[Automated testing]
+ If you like to keep up to speed, with latest developments and quality
+assurance, setting the environment variable NO_INTERACTION to 1, will not
+prompt the tester for any user input.
+Normally, the exit status of "make test" is zero, regardless of the results
+of independant tests. Set the environment variable REPORT_EXIT_STATUS to 1,
+to make "make test' set the exit statis ("$?") to non-zero, when a test has
+NOTE: the exit status of run-tests.php will be 1 when
+REPORT_EXIT_STATUS is set. The result of "make test" may be higher
+than that. At present, gmake 3.79.1 returns 2, so it is
+advised to test for non-zero, rather then a specific value.
[Creating new test files]

0 comments on commit 8bfb6c5

Please sign in to comment.
Something went wrong with that request. Please try again.