Skip to content

Automatic test diagnosis via expressions inspired by natural language

Choose a tag to compare

@rouson rouson released this 14 May 05:31
· 426 commits to main since this release
9fd8749

What's New

This release

  1. Defines idioms for writing compact expressions that construct test diagnoses that encapsulate automatically generated diagnostic output when tests fail and
  2. Reorganizes and enhances the example test suite and moves it a new demo subdirectory so that it builds as a separate fpm project with Julienne as a dependency.

The new operators eliminate much of the tedium of forming one's own diagnostics_string arguments when invoking test_diagnosis_t constructors as previously required.

An Expressive Idiom Constructing Test Diagnoses

For real or double precision values x, y, and tolerance and for integer values i and j, expressions of the following forms now evaluate to test_diagnosis_t objects constructed with descriptive diagnostic_string arguments tailored to the operators involved in the expressions:

x .approximates. y .within. tolerance
x .lessThan. y
x .greaterThan. y

i .equalsExpected. j
i .lessThan. j
i .greaterThan. j
i .lessThanOrEqualTo. j
i .greaterThanOrEqualTo. j

where all of the above operators are elemental and accept conformable arguments, i.e., same-shaped array operands or array/scalar operand combinations.

When the operands are arrays, the expressions evaluate to test_diagnosis_t arrays, which could be used as the result of a vector test function satisfying Julienne's vector_diagnosis_function_i abstract interface. Alternatively, array results can be aggregated into one test_diagnosis_t object using the .all. operator that this pull request also provides. For example,

.all. ([i,j] .lessThan. k)   ! test passes if i < k .and. j < k
.all. (i .greaterThan. [j,k]) ! test passes if i > j .and. i > k 
.all. ([i,j] .equalsExpected. [m,n]) ! test passes if i == m .and. j == n

where expressions operated on by .all. will result in diagnostic strings displaying only for the elements corresponding to test failures. See test/test_diagnosis_test_m.F90 for examples of the use of the above operators.

Equivalently, if pairwise aggregation suffices, then the user-defined .and. operator in this PR can be used as follows:

(2 .greaterThanOrEqualTo. expected_min) .and. (1 .equalsExpected. 1)

which is currently implemented using .all. with a size-2 array:

.all. ([lhs,rhs])

where lhs = 2 .greaterThanOrEqualTo. expected_min and rhs = 1 .equalsExpected. 1.

What's Changed

  • Chores by @rouson in #53
  • Chores and minor new feature by @rouson in #54
  • Feature: support comparison expressions that evaluate to test diagnoses by @rouson in #55
  • Reorganize examples and add string-handling example by @rouson in #59
  • fix(test_diagnosis): finish To Do item from PR #55 by @rouson in #61
  • doc(README): describe operators & improve structure by @rouson in #62
  • Revise example test suite by @rouson in #63

Full Changelog: 2.0.0...2.1.0