Prop.exists fails when used with ScalaTest matchers

[juanrh]

Hi,

I sent this to the ScalaCheck mailing list at google groups but it doesn't get published.
I have a problem about using ScalaTest matchers inside a Prop.exists property. When the property runs and a value is generated such that the matcher fails, it throws a TestFailedException and the property fails, instead of trying with the next generated value until reaching the limit in the number of tests. I guess this is because ScalaTest is designed around TestFailedExceptions, while ScalaCheck is designed around Prop objects. I have devised the following workaround, and I wanted to you what you think about it:

import org.scalacheck.{Properties, Gen}
import org.scalacheck.Prop.{forAll, exists, AnyOperators}
import org.scalacheck.Prop
import org.scalatest._
import org.scalatest.Matchers._
import scala.util.{Try, Success, Failure}
import org.scalacheck.util.Pretty

object ExistsBrokeTest extends Properties("Breaking Prop.exists with ScalaTest matchers") {
  property("forAll works ok with ScalaTest matchers, for successful tests") = 
    forAll { x : Int => 
      x + x should be (2 * x)
      true
    }

  property("forAll works ok with ScalaTest matchers, for failing tests") = 
    forAll { x : Int => 
      x + x should be (2 * x + 1)
      true
    }

  property("exists works ok without ScalaTest matchers") =
    exists ("x" |: Gen.choose(1, 20)) { x : Int =>
      x ?= 1
    }

  property("exists fails with ScalaTest matchers, for otherwise successful tests") =
    exists ("x" |: Gen.choose(0, 10)) { x : Int =>
      x should be (1)
      true
    }

  property("exists works ok with ScalaTest matchers wrapped by Try") = 
    exists ("x" |: Gen.choose(0, 10)) { x : Int =>
      Try(x should be (1)).isSuccess 
  }

  def safeProp[P <% Prop](p : => P) : Prop = {
    Try(p) match {
      case Success(pVal) => pVal
      case Failure(t) => t match {
        case _: TestFailedException => Prop.falsified
        case _ => Prop.exception(t) 
      }
    }
  }

  def safeExists[A, P](g: Gen[A])(f: (A) => P)
                      (implicit pv: (P) => Prop, pp: (A) => Pretty): Prop = {    
    exists(g)((x : A) => safeProp(pv(f(x))))(identity[Prop], pp)
    // This doesn't work for reasons unknown
    // exists(g)((x : A) => Prop.secure(pv(f(x))))(identity[Prop], pp)
  }  

  property("safeExists works ok with ScalaTest matchers, for successful tests") =
   safeExists("x" |: Gen.choose(0, 10)) { x : Int =>
     x should be (1)
     true
   }

  property("safeExists works ok with ScalaTest matchers, for failing tests") =
   safeExists("x" |: Gen.choose(0, 10)) { x : Int =>
     x should be (1000)
     true
   }
}

+ Breaking Prop.exists with ScalaTest matchers.forAll works ok with ScalaTe
  st matchers, for successful tests: OK, passed 100 tests.

! Breaking Prop.exists with ScalaTest matchers.forAll works ok with ScalaTe
  st matchers, for failing tests: Exception raised on property evaluation.
> ARG_0: -2147483648
> Exception: org.scalatest.exceptions.TestFailedException: 0 was not equal 
  to 1
org.scalatest.MatchersHelper$.newTestFailedException(MatchersHelper.scala:1
  60)
org.scalatest.Matchers$ShouldMethodHelper$.shouldMatcher(Matchers.scala:623
  1)
org.scalatest.Matchers$AnyShouldWrapper.should(Matchers.scala:6265)
es.ucm.fdi.sscheck.ExistsBrokeTest$$anonfun$2.apply$mcZI$sp(ExistsBrokeTest
  .scala:20)
es.ucm.fdi.sscheck.ExistsBrokeTest$$anonfun$2.apply(ExistsBrokeTest.scala:1
  9)

+ Breaking Prop.exists with ScalaTest matchers.exists works ok without Scal
  aTest matchers: OK, proved property.
> x: 1

! Breaking Prop.exists with ScalaTest matchers.exists fails with ScalaTest 
  matchers, for otherwise successful tests: Exception raised on property ev
  aluation.
> x: 3
> Exception: org.scalatest.exceptions.TestFailedException: 3 was not equal 
  to 1
org.scalatest.MatchersHelper$.newTestFailedException(MatchersHelper.scala:1
  60)
org.scalatest.Matchers$ShouldMethodHelper$.shouldMatcher(Matchers.scala:623
  1)
org.scalatest.Matchers$AnyShouldWrapper.should(Matchers.scala:6265)
es.ucm.fdi.sscheck.ExistsBrokeTest$$anonfun$3.apply$mcZI$sp(ExistsBrokeTest
  .scala:31)
es.ucm.fdi.sscheck.ExistsBrokeTest$$anonfun$3.apply(ExistsBrokeTest.scala:3
  0)

+ Breaking Prop.exists with ScalaTest matchers.exists works ok with ScalaTe
  st matchers wrapped by Try: OK, proved property.
> x: 1

+ Breaking Prop.exists with ScalaTest matchers.safeExists works ok with Sca
  laTest matchers, for successful tests: OK, proved property.
> x: 1

! Breaking Prop.exists with ScalaTest matchers.safeExists works ok with Sca
  laTest matchers, for failing tests: Gave up after only 0 passed tests. 10
  1 tests were discarded.

Also, I have noticed that org.scalatest.prop.GeneratorDrivenProperyChecks doesn't define exists, is there any technical reason for that? I would be glad to work on it if you think it is possible,

Thanks a lot in advance.

Greetings,

Juan Rodriguez Hortala

[juanrh]

Hi,

It's been almost a month, any thoughts on this?

[rickynils]

@juanrh Did you see Bill's reply in https://groups.google.com/d/msg/scalacheck/Ped7joQLhnY/gNH0SSWkKUgJ?

[juanrh]

Hi,

I hadn't seen that, thank you for the pointer. I'll close the issue then.

Thanks a lot for your help.

Greetings,

Juan