Move test-log logic for checks and tests inside their "around" parameters

[jackfirth]

Closes #70
Closes #9
Closes #14
Closes #58

This PR moves the test-log! logic of checks and test cases into the current-check-around and current-test-case-around parameters, respectively. Additionally, checks now parameterize current-check-around inside the body of define-check so that nested checks are evaluated as normal functions. This allows checks to be imperatively composed; they can be chained together in series just like normal exception-throwing validation functions. This involves several subtle changes to the effects and evaluation of checks:

Single nested check

(define-check (check-zero v)
  (check-equal? v 0))

(check-zero 0)

This now reports one test passing, and uses the default check info added by check-zero. Previously, it would report two tests passing.

Multiple nested checks

(define-check (check-zero-twice v)
  (check-equal? v 0)
  (check-equal? v 0))

(check-zero-twice 0)
(check-zero-twice 1)

The passing use reports one test passing, and the failing use reports one test failing with the check failure thrown by the first check-equal? expression. Previously, the passing use reported two tests passing and the failing use reported two tests failing and one test passing, as each of the nested checks would raise and swallow a check failure leading the outer check to believe nothing went wrong.

Multiple nested checks in test case

(test-case "passing test"
  (check-zero-twice 0))
(test-case "failing test"
  (check-zero-twice 1))

The passing test now reports one test passing, and the failing test reports one test failing. The passing test previously reported three tests passing, and the failing test previously reported one test passing and one test failing. It also only evaluated the first check inside check-zero-twice.

Multiple checks in a test case

(test-case "passing test"
  (check-equal? 1 1)
  (check-equal? 2 2))
(test-case "failing test"
  (check-equal? 1 1)
  (check-equal? 2 3))

The passing test now reports one test passing, and the failing test now reports one test failing. Previously the passing test reported two tests passing and the failing test reported one test passing and two tests failing.

Check in a test suite

For all test suites, calling run-tests on a suite now causes test-log! to report the same number of passing and failing tests as run-tests displays in its summary message. Previously, test-log! would report significantly more tests than run-tests claimed.

Other changes

• Some parameters now use contracts instead of manually checking types
• Some of rackunit's own tests are now plain test-case expressions instead of test-suite definitions provided and combined in all-rackunit-tests.rkt
• The rackunit/log tests now have slightly better failure messages
• Calls to test-log! now need to be explicitly made by the user of fold-test-results or foldts-test-results. This also allows rackunit test suites to be run without interacting with rackunit/log at all (which the GUI runner probably wants to do anyway)

Future work

This doesn't add any documentation of how rackunit checks behave when composed. It now mostly works "as expected", but without docs and tests guaranteeing this behavior it's not safe for users to rely on.

There are corresponding changes that need to be made in Typed Racket's rackunit wrapper code that duplicates rackunit's test case and test suite macros.

[bennn]

looks good

[bennn]

Does Typed Racket not depend on the return type of check-around?
(Or are you going to change TR before merging this?)

[jackfirth]

Checks created with define-check always returned void, so this shouldn't change the types. That's still the case according to the contract so these void expressions can be removed. I'll do that before merging.

[bennn]

remove this?

[jackfirth]

Removing it causes some tests to fail because (test-begin) was never a syntax error, but (parameterize ([param v])) is.

[bennn]

okay, maybe change to (void expr ....) ?

[jackfirth]

Turns out that doesn't work because the expr expressions actually aren't expressions, they could be definitions. I've added a comment and renamed the pattern variable to hopefully make this less confusing.

[jackfirth]

@bennn I'll merge this later today unless you've got any more comments. Thanks for the review!