defexpect 3-arg fails with alternate assertions

[NoahTheDuke]

Hey Sean!

I have a macro that creates some set-up and then calls expect on the results:

(defmacro expect-match
  ([expected s] `(expect-match ~expected ~s nil))
  ([expected s config]
   `(let [diagnostics# (check-all ~s ~config)]
      (expect (~'match? ~expected diagnostics#)))))

This works perfectly when I put it in defexpect calls, except when I put two of them in a row with no other calls:

(ns noahtheduke.splint.expectations-test
  (:require
   [expectations.clojure.test :refer [defexpect]]
   [noahtheduke.splint.test-helpers :refer [expect-match]]))

(defexpect example-test
  (expect-match nil "#_:splint/disable (+ x 1)")
  (expect-match nil "(+ 1 x)"))

If they both pass, then it says there's 3 passing tests, and if one of them fails, it shows two failures, the second being the mismatch between the first assertion and the second assertion:

Testing noahtheduke.splint.expectations-test

FAIL in (example-test) (NO_SOURCE_FILE:8)
expected: (match? nil diagnostics__28464__auto__)
  actual: (mismatch
 (expected nil)
 (actual ({...})))


FAIL in (example-test) (NO_SOURCE_FILE:6)
(expect-match nil "(+ 1 x)")

expected: (=? (expect-match nil "#_:splint/disable (+ x 1)") (expect-match nil "(+ 1 x)"))
  actual: (not (=? true false))

Ran 1 tests containing 3 assertions.
2 failures, 0 errors.
{:test 1, :pass 1, :fail 2, :error 0, :type :summary}

Is there a way to avoid this behavior besides using plain deftest? I don't mind doing that, but it's nice when I can :require the one library instead of both (having to pay attention to when I have exactly two assertions in a form).

[seancorfield]

If you add another form, like :ignore or something, then the special handling of two expressions will not kick in.

This is due to Expectations continuing to support (defexpect my-test expected-form actual-form) -- it walks the two expressions and if they don't explicitly contain (expect e a) forms, then it assumes you intended this as a short-hand for (defexpect my-test (expect expected-form actual-form))

This has been a bit of a bugbear for me to continue supporting, but there are test suites out there that rely on it... I'll have a think about making it smarter... Perhaps matching any symbol starting with expect and assuming it's a test...

[seancorfield]

Can you check that the latest develop version solves this for you (by treating your expect-match as an "Expectation")?

If so, I'll make a new release.

[NoahTheDuke]

That works for my use-case, thanks.