More documentation for processing tests

Author: m-kuhn

python/plugins/processing/tests/README.md

@@ -11,13 +11,11 @@ A basic test appears under the toplevel key `tests` and looks like this:
     algorithm: qgis:polygoncentroids
     params:
       - type: vector
-        location: qgs
-        name: polys.shp
+        name: polys.gml
     results:
-      - id: OUTPUT_LAYER
+       OUTPUT_LAYER:
         type: vector
-        location: proc
-        name: polys_centroid.geojson
+        name: expected/polys_centroid.gml
 
 How To
 ------
@@ -30,13 +28,11 @@ geometry types and good readability. Redirect output to
 `python/plugins/processing/tests/testdata/expected`
 
 When you have run the algorithm, go to "Processing" > "History" and find the
-algorithm which you have just run. This looks like
-
-    processing.runalg("qgis:densifygeometries","/home/mku/dev/cpp/qgis/QGIS/tests/testdata/polys.shp",2,"/home/mku/dev/cpp/qgis/QGIS/python/plugins/processing/tests/testdata/polys_densify.geojson")
+algorithm which you have just run. Right click the algorithm and click "Create test".
+A new window will open with a text definition.
 
 Open the file `python/plugins/processing/tests/testdata/algorithm_tests.yaml`,
-copy an existing test block and adjust it to your needs based on the
-information found in the history.
+copy the text definition there.
 
 The first string from the command goes to the key `algorithm`, the subsequent
 ones to params and the last one(s) to results.
@@ -47,72 +43,110 @@ The above translates to
       algorithm: qgis:densifygeometriesgivenaninterval
       params:
         - type: vector
-          location: qgs
-          name: polys.shp
+          name: polys.gml
         - 2 # Interval
       results:
-        - id: OUTPUT
+        OUTPUT:
           type: vector
-          location: proc
-          name: expected/polys_densify.geojson
+          name: expected/polys_densify.gml
 
 Params and results
 ------------------
 
-Trivial type parameters
-.......................
+### Trivial type parameters
+
+Params and results are specified as lists or dictionaries:
 
-Params and results are specified as lists:
+    params:
+      INTERVAL: 5
+      INTERPOLATE: True
+      NAME: A processing test
+      
+or
 
     params:
       - 2
       - string
       - another param
 
-As in the example above they can be plain variables.
-
-Layer type parameters
-.....................
+### Layer type parameters
 
-To specify layers you will have to specify
+You will often need to specify layers as parameters. To specify a layer you will need to specify:
 
  * the type
    * `vector` or `raster`
- * a location to allow using files from the shared qgis test data
-   * `qgs` will look for the file in the src/tests/testdata
-   * `proc` will look for the file in python/plugins/processing/tests/testdata
-     you should use this location for expected data.
  * a name
-   * relative path like `expected/polys_centroid.geojson`
+   * relative path like `expected/polys_centroid.gml`
 
     params:
-      - 2
-      - string
-      - type: vector
-        location: qgs
-        name: polys.shp
-      - another param
+      PAR: 2
+      STR: string
+      LAYER:
+        type: vector
+        name: polys.gml
+      OTHER: another param
 
-Results
-.......
+### Results
 
-Results have a special key `id` which is required because an algorithm can
-produce multiple results. If you don't know the `id`, just start with `OUTPUT`
-and run the test. You will be told if it was wrong and about the possible
-values.
+Results are specified very similar.
 
-To deal with a certain tolerance for output values you can specify a
-`compare` property for an output.
+#### Basic vector files
+
+It couldn't be more trivial
+
+    OUTPUT:
+      name: expected/qgis_intersection.gml
+      type: vector
+
+#### Vector with tolerance
 
-For a vector layer this means
+Sometimes different platforms create slightly different results which are
+still acceptable. In this case (but only then) you may also use additional
+properties to define how exactly a layer is compared.
+
+To deal with a certain tolerance for output values you can specify a
+`compare` property for an output. The compare property can contain sub-properties
+for `fields`. This contains information about how precisely a certain field is
+compared (`precision`) or a field can even entirely be `skip`ed. There is a special
+field name `__all__` which will apply a certain tolerance to all fields.
+There is another property `geometry`  which also accepts a `precision` which is
+applied to each vertex.
 
     OUTPUT:
       type: vector
       name: expected/abcd.geojson
-    compare:
-      fields:
-        __all__:
-          precision: 5 # compare to a precision of .00001 on all fields
-        A: skip # skip field A
-      geometry:
-        precision: 5 # compare coordinates with a precision of 5 digits
+      compare:
+        fields:
+          __all__:
+            precision: 5 # compare to a precision of .00001 on all fields
+          A: skip # skip field A
+        geometry:
+          precision: 5 # compare coordinates with a precision of 5 digits
+
+#### Raster files
+
+Raster files are compared with a hash checksum. This is calculated when you create
+a test from the processing history.
+
+    OUTPUT:
+      type: rasterhash
+      hash: f1fedeb6782f9389cf43590d4c85ada9155ab61fef6dc285aaeb54d6
+      
+#### Files
+
+You can compare the content of an ouptut file by an expected result reference file
+
+    OUTPUT_HTML_FILE:
+      name: expected/basic_statistics_string.html
+      type: file
+
+Or you can use one or more regular expressions that will be [matched](https://docs.python.org/2/library/re.html#re.search) against the file
+content
+
+    OUTPUT:
+      name: expected/gdal/layer_info.html
+      type: regex
+      rules:
+        - 'Extent: \(-1.000000, -3.000000\) - \(11.000000, 5.000000\)'
+        - 'Geometry: Line String'
+        - 'Feature Count: 6'