Commands sequence shrinking not working (since scala 2.12 update)

[beatkyo]

I have been investigating the way command failures are reported because it seemed to me that shrinking was not working. I may have spotted a bug introduced with scala 2.12. Here is how to reproduce the behavior I observed:

Apply this diff to be able to see the failure:

diff --git a/jvm/src/test/scala/org/scalacheck/commands/CommandsSpecification.scala b/jvm/src/test/scala/org/scalacheck/commands/CommandsSpecification.scala
index c60e276..7e66cb1 100644
--- a/jvm/src/test/scala/org/scalacheck/commands/CommandsSpecification.scala
+++ b/jvm/src/test/scala/org/scalacheck/commands/CommandsSpecification.scala
@@ -13,13 +13,13 @@ import org.scalacheck._

 object CommandsSpecification extends Properties("Commands") {

-  property("commands") = TestCommands.property(threadCount = 4)
+  property("commands") = TestCommands.property(threadCount = 1)

   object TestCommands extends Commands {
     case class Counter(var n: Int) {
       val lock = new java.util.concurrent.locks.ReentrantLock
       def inc = { lock.lock; n += 1; lock.unlock; n }
-      def dec = { lock.lock; n -= 1; lock.unlock; n }
+      def dec = { lock.lock; if(n < 90) n -= 1; lock.unlock; n }
       //def inc = { n += 1; n }
       //def dec = { n -= 1; n }
       def get = n

In sbt run the following commands and notice the reported seqcmds:

scala 2.12.6

scalacheck >> ++ 2.12.6 jvm/testOnly org.scalacheck.commands.CommandsSpecification
[info] Setting Scala version to 2.12.6 on 3 projects.
[info] Excluded 1 projects, run ++ 2.12.6 -v for more details.
[info] Reapplying settings...
[info] Set current project to scalacheck (in build file:/C:/Users/dalexandre/dev/scalacheck/)
[info] ! Commands.commands: Falsified after 8 passed tests.
[info] > Labels of failing property:
[info] initialstate = 93
[info] seqcmds = (Inc => 94; Get => 94; Inc => 95; Inc => 96; Inc => 97; Inc => 98; Dec => 98; Inc => 99)
[info] > ARG_0: Actions(93,List(Inc, Get, Inc, Inc, Inc, Inc, Dec, Inc),List())
[info] Failed: Total 1, Failed 1, Errors 0, Passed 0
[error] Failed tests:
[error]         org.scalacheck.commands.CommandsSpecification
[error] (jvm / Test / testOnly) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 0 s, completed 2 janv. 2019 15:57:52

scala 2.11.12

scalacheck >> ++ 2.11.12 jvm/testOnly org.scalacheck.commands.CommandsSpecification
[info] Setting Scala version to 2.11.12 on 4 projects.
[info] Reapplying settings...
[info] Set current project to scalacheck (in build file:/C:/Users/dalexandre/dev/scalacheck/)
[info] ! Commands.commands: Falsified after 3 passed tests.
[info] > Labels of failing property:
[info] initialstate = 99
[info] seqcmds = (Dec => 99)
[info] > ARG_0: Actions(99,List(Dec),List())
[info] > ARG_0_ORIGINAL: Actions(99,List(Inc, Dec, Inc),List())
[info] Failed: Total 1, Failed 1, Errors 0, Passed 0
[error] Failed tests:
[error]         org.scalacheck.commands.CommandsSpecification
[error] (jvm / Test / testOnly) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 0 s, completed 2 janv. 2019 16:01:28

In 2.11.x sequence has been shrunk to one command whereas in 2.12.x the sequence is left untouched. This may be related to #426 concerns as well.

[Leonti]

Can confirm that shriking is not occurring for me - I'm trying out the example from 2014 presentation:

package example

import org.scalacheck.{Gen, Prop, Shrink}
import org.scalacheck.commands.Commands
import org.scalatest.{FlatSpec, Matchers}
import org.scalatest.prop.Checkers
import org.scalacheck.Arbitrary.arbitrary
import org.scalacheck.Gen.oneOf

import scala.util.{Success, Try}

case class Counter(private var n: Int = 0) {
  def increment(): Int = {
    n = n + 1
    n
  }
  def get: Int = n
}

object StatefulCommands extends Commands {
  case class State(count: Long)

  override type Sut = Counter

  override def canCreateNewSut(newState: State, initSuts: Traversable[State], runningSuts: Traversable[Sut]): Boolean = true

  override def newSut(state: State): Sut = Counter(state.count.toInt)

  override def destroySut(sut: Sut): Unit = ()

  override def initialPreCondition(state: State): Boolean = true

  override def genInitialState: Gen[State] = arbitrary[Int].map(n => State.apply(n.toLong))

  override def genCommand(state: State): Gen[Command] = oneOf(Increment, Get)

  override def shrinkState: Shrink[State] = Shrink(x => implicitly[Shrink[Long]].shrink(x.count).map(State.apply))

  case object Increment extends Command {
    override type Result = Int

    override def run(counter: Counter): Result = counter.increment

    override def nextState(state: State): State = state.copy(count = state.count + 1)

    override def preCondition(state: State): Boolean = true

    override def postCondition(state: State, result: Try[Result]): Prop = result == Success(state.count + 1)
  }

  case object Get extends Command {
    override type Result = Int

    override def run(counter: Counter): Result = counter.get

    override def nextState(state: State): State = state

    override def preCondition(state: State): Boolean = true

    override def postCondition(state: State, result: Try[Result]): Prop =
      result == Success(state.count)
  }

}

class StatefulSpec extends FlatSpec with Matchers with Checkers {

  it should "successfully evaluate commands" in {
    check(StatefulCommands.property())
  }
}

This is the output:

  Occurred when passed generated values (
    arg0 = Actions(State(2147483647),List(Get, Increment, Increment, Increment, Increment, Increment, Increment, Get, Increment, Increment, Increment, Increment, Increment, Get, Increment, Increment, Get, Increment, Get, Increment, Increment, Get, Get, Increment, Get, Increment, Increment, Increment, Get, Increment),List())
  )
  Label of failing property:
    initialstate = State(2147483647)
seqcmds = (Get => 2147483647; Increment => -2147483648; Increment => -2147483647; Increment => -2147483646; Increment => -2147483645; Increment => -2147483644; Increment => -2147483643; Get => -2147483643; Increment => -2147483642; Increment => -2147483641; Increment => -2147483640; Increment => -2147483639; Increment => -2147483638; Get => -2147483638; Increment => -2147483637; Increment => -2147483636; Get => -2147483636; Increment => -2147483635; Get => -2147483635; Increment => -2147483634; Increment => -2147483633; Get => -2147483633; Get => -2147483633; Increment => -2147483632; Get => -2147483632; Increment => -2147483631; Increment => -2147483630; Increment => -2147483629; Get => -2147483629; Increment => -2147483628)

The same example from presentation got shrunk to a single command.

[nrinaudo]

This seems to be linked to implicit resolution rules changes between 2.11 and 2.12.

It's fixed by declaring shrinkActions before property in Commands, but it's not clear to me why that'd be the case. Happy to submit a PR for that, but can anyone explain why this fixes things?