Nested Specs in user scopes don't run

[elliottneilclark]

@RunWith(classOf[JUnitRunner])
class FetcherSpec extends Specification {

  "HBaseFetcher" should {

    "get data from a running test cluster" in new TestClusterSpec  {
      "get an empty list for users not in hbase" in  {
        1 must_== 2
      }
      "get a list of one for auto-generated users" in  {
        2 must_==1
      }

    }
  }
}

The above passes where TestClusterSpec is a BeforeAfter (I've also tried it with scope). The before and after methods are called but the tests are never run.

[etorreborre]

Scopes cannot be used to do what you want to achieve. Their only purpose is to execute some code inside of their body and throw exceptions, like FailureException if there is a failure.

You can however do this:

import org.specs2.mutable.Specification
import org.specs2.specification.SpecificationStructure

class TestSpec extends Specification { outer =>
  "HBaseFetcher" should {
    br
    "get data from a running test cluster" >> nest {
      new Nested  {
        "then run more nested tests" >> {
          "get an empty list for users not in hbase" in ok
          "get a list of one for auto-generated users" >> ok
        }
      }
    }
    "and run other tests" >> ok
  }
  def nest(spec: SpecificationStructure) = {
    addFragments { spec.is.middle }
    t
  }
}
class Nested extends Specification {
  "Run the first nested test" >> ok
}

This will output:

[info] TestSpec
[info] 
[info] HBaseFetcher should
[info]  
[info]   get data from a running test cluster
[info]   + Run the first nested test
[info]     then run more nested tests
[info]     + get an empty list for users not in hbase
[info]     + get a list of one for auto-generated users
[info]  
[info]  
[info]   + and run other tests
[info]  
[info]  
[info] Total for specification TestSpec
[info] Finished in 20 ms
[info] 4 examples, 0 failure, 0 error

I can add the nest method to the mutable.Specification trait. In this case I'll try to remove the superflous newline after the specification has been nested.

[elliottneilclark]

Thanks.
A nest would be really useful. Trying to get something like a beforeAll and afterAll for a group of specifications is really useful.

[etorreborre]

Note that you can already use include to include specifications inside another one. The difference with nest is that the output will show the beginning and the stats for the included specification.

Then, if you want to add a beforeAll/afterAll behavior you can use steps like that:

class HBaseFetcher extends Specification {
   step(println("before all"))

   // you can use a text fragment to introduce the included specifications
  "HBASE SPECIFICATIONS".txt
   include(spec1, spec2, spec3)

   step(println("after all"))
}

[elliottneilclark]

Nest + Step gets me exactly what I was looking for. Thanks so much.

[etorreborre]

I published a new 1.12-SNAPSHOT where the nest method is actually called inline because it better conveys the idea that only the "middle" fragments of a specification are included. One important point though, if you have specific arguments in the specifications you want to inline you will have to use include because this only will preserve the effect of those arguments on the included fragments:

class Spec1 extends Specification {
   // all the sub spec examples will be executed concurrently
   inline(new SubSpec)

   // all the sub spec examples will be executed sequentially
   include(new SubSpec)
}
class SubSpec extends Specification { sequential }

[blueskymonster]

Hi, this thread was great in helping me understand how to get nested tests working, but I'm having trouble getting it to work with the inMemoryDatabase configuration in Play 2.1.1 (let me know if this is the wrong place to ask this question, but it seemed like the most relevant place).

I am able to run a non-nested tests with the inMemoryDatabase configuration as follows with no issues:

"Test with DB" in new WithApplication(FakeApplication(additionalConfiguration = inMemoryDatabase())) {
    // tests that access the DB
}

and I'm able to run nested tests as described above by @etorreborre without the in memory db, but I can't figure out how to combine the two. When I attempt to combine them, I've been getting the following error:

[error] Could not run test UserSpec: Configuration error: Configuration error[Cannot connect to database [default]]
Throwable escaped the test run of 'UserSpec': Configuration error: Configuration error[Cannot connect to database [default]]
Configuration error: Configuration error[Cannot connect to database [default]]

My attempt looks analogous to this:

import org.specs2.mutable.Specification
import org.specs2.specification.SpecificationStructure

class TestSpec extends Specification { outer =>
  "Test" should {
    br
    "Prepare for testing with db" >> nest {
      new NestedWithDB  {
        "then run nested tests" >> {
          "do something with the db" in {
            // tests that access db
          }
        }
      }
    }
    "and run other tests" >> ok
  }
  def nest(spec: SpecificationStructure) = {
    addFragments { spec.is.middle }
    t
  }
}
class NestedWithDB extends WithApplication(FakeApplication(additionalConfiguration = inMemoryDatabase())) with Specification {
    "Initialize Fake DB" >> ok
}

It's strange to me that the configuration would work fine in the one case but not the other...
I'm at my wit's end trying to get the two to play nicely and have a feeling that I'm missing something obvious, but I'm not sure what it is...

[etorreborre]

When you write in new WithApplication you create an anonymous class which will implement the mutable.Around trait. This trait makes sure that everything that is executed inside the body of the anonymous class is executed in the context of a fake application.

However, when you write class Nested extends WithApplication with Specification you only provide an application context for the creation of examples inside the nested specification, not their execution.

The right thing to do, I think, would be to use the FixtureExample trait introduced in specs2 2.0 and do the following:

trait FakeApplicationFixture extends FixtureExample[FakeApplication] {
  type FA = FakeApplication

  def fixture[R : AsResult](f: FA => R) = {
    val app = FakeApplication(additionalConfiguration = inMemoryDatabase())
    new WithApplication(app)(f(app))
  }
}

class TestSpec extends Specification with FakeApplicationFixture {
"Test" should {
    br
    "Prepare for testing with db" >> nest {
      new NestedWithDB  {
        "then run nested tests" >> {
          "do something with the db" in { (app: FA) =>
            // tests that access db
          }
        }
      }
    }
    "and run other tests" >> { application: FA => ok }
  }
  def nest(spec: SpecificationStructure) = {
    addFragments { spec.is.middle }
    t
  } 
}
class NestedWithDB extends Specification with FakeApplicationFixture {
    "Initialize Fake DB" >> { app: FA => ok }
}

This way, you make sure that all examples are executed inside a FakeApplication context and you can also access the application variables inside a test with the app variable.

The code above is just a sketch and will probably not compile as it is. Please experiment with it and tell me if you couldn't make it compile.

[blueskymonster]

Hi Eric @etorreborre , thanks for the quick and detailed response!

Unfortunately, I cannot upgrade to the newest version of specs2, so I'm trying to pull out the appropriate snippets and include them in my codebase. I have this so far:

trait Fixture[T] {
  def apply[R : AsResult](f: T => R): Result
}

object Fixture {
  implicit def fixtureHasMonad: Monad[Fixture] = new Monad[Fixture] {
    def point[A](a: =>A) = new Fixture[A] {
      def apply[R : AsResult](f: A => R): Result = AsResult(f(a))
    }
    def bind[A, B](fixture: Fixture[A])(fa: A => Fixture[B]): Fixture[B] = new Fixture[B] {
      def apply[R : AsResult](fb: B => R): Result = fixture((a: A) => fa(a)(fb))
    }
  }
}

trait FixtureExample[T] {
  protected def fixture[R : AsResult](f: T => R): Result
  implicit def fixtureContext = new Fixture[T] { def apply[R : AsResult](f: T => R) = fixture(f) }
}

trait FakeApplicationFixture extends FixtureExample[FakeApplication] {
  type FA = FakeApplication

  def fixture[R: AsResult](f: FA => R) = {
    val app = FakeApplication(additionalConfiguration = inMemoryDatabase())
    new WithApplication(app)(f(app)) {} // Error on this line: Expression of type WithApplication doesn't conform to expected type Result
  }
}

but it doesn't quite compile... I've marked the line that generates the error. I add the curly braces because WithApplication is an abstract class so it need to be instantiated.

One more thing, if I want to include code to be run before every "String" >> { app: FA => ok } scope, where should that be included?

[etorreborre]

The code doesn't compile because it needs an implicit definition which is found in the FragmentsBuilder trait:

  implicit def inScope(s: Scope): Success = Success()

I think that it will work with a self-type:

trait FakeApplicationFixture extends FixtureExample[FakeApplication] { self: FragmentsBuilder =>

One more thing, if I want to include code to be run before every "String" >> { app: FA => ok } scope,
where should that be included?

You can put that code right before the new WithApplication:

trait FakeApplicationFixture extends FixtureExample[FakeApplication] { self: FragmentsBuilder =>
  type FA = FakeApplication
  // override to add some "before" behaviour
  def before {}

  // override to modify the application
  def application = FakeApplication(additionalConfiguration = self.additionalConfiguration)
  def additionalConfiguration =  inMemoryDatabase()

  def fixture[R: AsResult](f: FA => R) = {
    before
    new WithApplication(app)(f(application)) {} 
  }
}

[blueskymonster]

@etorreborre , I'm sorry to keep bugging you, but you've really been a great help so far. Hopefully this is the last issue to iron out. I feel like we're almost there.

When you refer to FragmentsBuilder, I'm assuming you're referring to the FragmentsBuilder in the mutable library...

Adding that bit in fixes the issue with FakeApplicationFixture, but leaves me with an error when I try to use it:
illegal inheritance;
[error] self-type FakeApplicationFixture does not conform to FakeApplicationFixture's selftype FakeApplicationFixture with org.specs2.mutable.FragmentsBuilder
[error] "test some struff" >> new FakeApplicationFixture {
[error] ^

I've always found illegal inheritance errors to be among the least informative errors in Scala's compiler. Without a deep knowledge of the code base you're trying to extend, it is very difficult to understand where the error lies. For a bit of context on the code generating this error, here's a snippet which is equivalent in structure to my code:

trait Fixture[T] {
  def apply[R : AsResult](f: T => R): Result
}

object Fixture {
  implicit def fixtureHasMonad: Monad[Fixture] = new Monad[Fixture] {
    def point[A](a: =>A) = new Fixture[A] {
      def apply[R : AsResult](f: A => R): Result = AsResult(f(a))
    }
    def bind[A, B](fixture: Fixture[A])(fa: A => Fixture[B]): Fixture[B] = new Fixture[B] {
      def apply[R : AsResult](fb: B => R): Result = fixture((a: A) => fa(a)(fb))
    }
  }
}

trait FixtureExample[T] {
  protected def fixture[R : AsResult](f: T => R): Result
  implicit def fixtureContext = new Fixture[T] { def apply[R : AsResult](f: T => R) = fixture(f) }
}

trait FakeApplicationFixture extends FixtureExample[FakeApplication] { self: FragmentsBuilder =>
  type FA = FakeApplication

  def application = FakeApplication(additionalConfiguration = this.additionalConfigurations)
  def additionalConfigurations = inMemoryDatabase()

  def fixture[R: AsResult](f: FA => R) = {
    new WithApplication(application)(f(application)) {}
  }
}

class TestSpec extends Specification {
  "Test Class" should {
    br
    "test some stuff" >> new FakeApplicationFixture {
      None must beNone
    }
  }
}