Docs: add document "writing-tests" also to 2.9

docs/po4a.cfg

@@ -26,6 +26,7 @@
 [type: AsciiDoc_def] src/code/nml-messages.adoc $lang:src/$lang/code/nml-messages.adoc
 [type: AsciiDoc_def] src/code/rs274.adoc $lang:src/$lang/code/rs274.adoc
 [type: AsciiDoc_def] src/code/style-guide.adoc $lang:src/$lang/code/style-guide.adoc
+[type: AsciiDoc_def] src/code/writing-tests.adoc $lang:src/$lang/code/writing-tests.adoc
 [type: AsciiDoc_def] src/common/emc-history.adoc $lang:src/$lang/common/emc-history.adoc
 [type: AsciiDoc_def] src/common/glossary.adoc $lang:src/$lang/common/glossary.adoc
 [type: AsciiDoc_def] src/common/gpld-copyright.adoc $lang:src/$lang/common/gpld-copyright.adoc

docs/src/Submakefile

@@ -41,6 +41,7 @@ DOC_SRCS_EN := \
 	code/adding-configs.adoc \
 	code/contributing-to-linuxcnc.adoc \
 	code/building-linuxcnc.adoc \
+	code/writing-tests.adoc \
 	common/emc-history.adoc \
 	common/glossary.adoc \
 	common/gpld-copyright.adoc \

docs/src/code/contributing-to-linuxcnc.adoc

@@ -112,7 +112,7 @@ like this:
 * Communicate with the project developers and let us know what you're hacking on.  Explain what you are doing, and why.
 * Clone the git repo.
 * Make your changes in a local branch.
-* Adding documentation and tests is an important part of adding a new feature.  Otherwise, others won't know how to use your feature, and if other changes break your feature it can go unnoticed without a test.
+* Adding documentation and <<cha:writing-tests,writing tests>> is an important part of adding a new feature.  Otherwise, others won't know how to use your feature, and if other changes break your feature it can go unnoticed without a test.
 * Share your changes with the other project developers in one of these ways:
 ** Push your branch to github and create a github pull request to <https://github.com/linuxcnc/linuxcnc> (this requires a github account), or
 ** Push your branch to a publicly visible git repo (such as github, or your own publicly-accessible server, etc) and share that location on the emc-developers mailing list, or

docs/src/code/writing-tests.adoc

@@ -0,0 +1,124 @@
+[[cha:writing-tests]]
+= Writing automated LinuxCNC tests
+
+== The test framework
+
+The code base has unit and integration tests that can be executed
+automatically to ensure the program work as intended.  Such tests are
+often written to trigger a bug and to ensure the bug is detected if it
+resurface in the future, but also to validate behavior of components
+and interfaces.
+
+The tests are collected in the `tests/` directory.  The individual tests
+are in subdirectories of this directory. The tests are grouped in
+directories.
+
+== Running tests
+
+The tests are executed by the `scripts/runtests` script generated from
+`scripts/runtests.in` during build. The runtests script will by default
+locate tests to run under `tests/`, but can be limited to only run a
+limited set of tests by specifying the directory of the test or tests
+as argument(s).
+
+Here is an example running only the tests in `tests/lathe/`
+
+----
+$ scripts/runtests tests/lathe/
+Running test: tests/lathe
+Runtest: 1 tests run, 1 successful, 0 failed + 0 expected, 0 skipped
+----
+
+The runtests script looks for all files named _test_, _test.sh_ and
+_test.hal_ below the directories specified on the command line, or under
+`tests/` if no command line argument is specified. These files
+represent three different ways to run the tests.
+
+The _runtests_ script accept the following arguments, see the output
+from `scripts/runtests -h` for the authorative list:
+----
+-n  do not remove temporary files for successful tests.
+-s  stop after any failed test.
+-p  print stderr and result files.
+-c  Remove temporary files from an earlier test run.
+-u  Only run tests that require normal user access.
+-v  Show stdout and stderr (normally it's hidden).
+----
+== Writing tests
+
+Make sure the test can run successfully without a working X11 display,
+i.e. with the DISPLAY environment variable unset.
+
+1. Create a folder in `tests/`.
+2. Provide one test script.
+3. Evaluate the output with one of the options below.
+
+These are the files considered in the directory with the individual
+tests:
+
+.Test script (only one of these three)
+
+test::
+   A program executed and its exit code and output is checked using
+   either checkresult or expected.
+
+test.sh::
+   A bash script executed and its exit code and output is checked using
+   either checkresult or expected.
+
+test.hal::
+   A HAL script executed using `halrun -f test.hal` and its exit code
+   and output is checked using either checkresult or expected.
+
+.Test evaluation
+
+ expected::
+   A file whos content is compared to the output from running the test
+   scripts.  If the test output is identical to the content of the
+   expected file, the test succeed.
+
+checkresult::
+   An excutable file to perform more complex validation than just comparing
+   the output of a test script. It gets the filename of the test program as
+   its command line argument. The exit code of this program controls the result
+   of the test. If both `expected` and `checkresult` exist, only `checkresult`
+   is consulted to validate the test output.
+
+ xfail::
+   If this file exist, a test failure is expected and do not cause
+   runtests to return an exit code signaling an error.
+
+ skip::
+   If this file exist, the test is skipped and not executed at all.
+
+ control::
+   This file can be used to flag specific needs in the test.  At the
+   moment, the use of _sudo_ can be flagged, and tests requiring sudo
+   can be skipped when using `runtests -u`. To flag such requirement,
+   add a line with `Restrictions: sudo` to this file.
+
+== Some testing approaches
+
+There are various ways to structure a test, depending on what one want
+to test. Here are a few ideas on how to do it.
+
+=== Non-interactive "GUI"
+
+If you want to test some operations in the user interface, a useful
+approach is is to write a custom "GUI" simulating the operations.
+This can be done by creating a normal LinuxCNC setup and pointing the
+[DISPLAY] DISPLAY value to a script that do the operations needed to
+test the behaviour.
+
+Examples of this approach can be found in `tests/halui/jogging/` and
+`tests/interp/subroutine-return/`.
+
+=== Recording HAL pin transitions
+
+Using the _sampler_ and _halsampler_ HAL components, one can set up a
+HAL configuration and collect pin value settings and changes and dump
+the result to stdout (or a file).  The end result can then be compared
+with the expected output to verify if HAL behave as expected.
+
+Examples of this approach can be found in `tests/multiclick/` and
+`tests/stepgen.2/`.

docs/src/index.tmpl

@@ -316,6 +316,7 @@ function setup_page(){
 		<li><a href="code/style-guide.html">Style Guide</a></li>
 		<li><a href="code/adding-configs.html">Adding Configs for Selection</a></li>
 		<li><a href="code/contributing-to-linuxcnc.html">Contributing to LinuxCNC</a></li>
+		<li><a href="code/writing-tests.html">Writing tests for LinuxCNC</a></li>
 		<li><a href="code/building-linuxcnc.html">Building LinuxCNC</a></li>
 	</ul>
 </div>