Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New API and DSL #78

Merged
merged 2 commits into from
Nov 18, 2019
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
127 changes: 127 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,8 +254,135 @@ class MysqlSpec extends FlatSpec with ForAllTestContainer {
}
```

## New API

Starting from 0.34.0 version testcontainers-scala provides the new API.
The main motivation points are in the [pull request](https://github.com/testcontainers/testcontainers-scala/pull/78).

### `Container` and `ContainerDef`

Docker containers are represented through the two different entities:
1. `ContainerDef` — it's container definition. `ContainerDef` describes, how to build a container.
You can think about it like about a container constructor, or dockerfile description.
Usually, `ContainerDef` receives some parameters.
`ContainerDef` has a `start()` method. It returns a started `Container`.
2. `Container` — it's a started container. You can interact with it through its methods.
For example, in the case of `MySQLContainer` you can get it's JDBC URL with `jdbcUrl` method.
`Container` is the main entity for using inside tests.

### Scalatest usage

You can use one of the four traits:
1. `TestContainerForAll` — will start a single container before all tests and stop after all tests.
2. `TestContainerForEach` — will start a single container before each test and stop after each test.
3. `TestContainersForAll` — will start multiple containers before all tests and stop after all tests.
4. `TestContainersForEach` — will start multiple containers before each test and stop after each test.


#### Single container in tests

If you want to use a single container in your test:
```scala
class MysqlSpec extends FlatSpec with TestContainerForAll {

// You need to override `containerDef` with needed container definition
override val containerDef = MySQLContainer.Def()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still believe that main container class should be its definition.

override val container = MySQLContainer()

to make it more intuitive

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I played with this a bit and I didn't come up with something reasonable enough. Going this way will make it much harder to maintain backward compatibility with the current API. In this pull request, I managed to maintain backward compatibility relatively easy.

Anyway, this is experimental. I suggest to try it and wait for feedback.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. I think I need to play with the code on my local machine to get a better sense of the backward compatibility in this implementation.

Btw, do you know any good way to gather feedback from early adopters?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I definitely will promote these changes in the Russian speaking scala user group in the telegram. I know that some people there are using testcontainers.

Also, recently I spoke about testcontainers and this pull request in the scalalaz podcast. This episode is still not released though. We need to wait for it a few more days.

I hope it would be enough at the start.


// To use containers in tests you need to use `withContainers` function
it should "test" in withContainers { mysqlContainer =>
// Inside your test body you can do with your container whatever you want to
assert(mysqlContainer.jdbcUrl.nonEmpty)
}
}
```

Usage of `TestContainerForEach` is not different from the example above.

#### Multiple containers in tests

If you want to use multiple containers in your test:
```scala
class ExampleSpec extends FlatSpec with TestContainersForAll {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it be possible to replace abstract type with a type parameter?

class ExampleSpec extends FlatSpec with TestContainersForAll[MySQLContainer and PostgreSQLContainer]

I think this way it would be less verbose

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At first test traits had type parameters, as you suggest. But I changed it to the type member. There are 3 reasons:

  1. You can't refer to a type parameter inside your test body. This is important, because startContainers returns Containers type, and withContainers receives a function with type Containers => Unit.
  2. If you have a lot of containers it would looks weird in my opinion. I have a service with 6 or 7 containers in testcontainers tests, so this is the real situation.
  3. I found that type member usage is a bit more explicit. For example:
    class ExampleSpec extends FlatSpec with TestContainersForAll[MySQLContainer and PostgreSQLContainer] {
      override def startContainers(): Containers = {
        val container1 = MySQLContainer.Def().start ()
        val container2 = PostgreSQLContainer.Def().start ()
        container1 and container2
      }
      
      it should "test" in withContainers { case mysqlContainer and pgContainer =>
        assert(mysqlContainer.jdbcUrl.nonEmpty && pgContainer.jdbcUrl.nonEmpty)
      }
      
    }
    For me, it looks like MySQLContainer and PostgreSQLContainer are lost inside test definition. But here they are a lot more visible and explicit:
    class ExampleSpec extends FlatSpec with TestContainersForAll {
    
      override type Containers = MySQLContainer and PostgreSQLContainer
    
      override def startContainers(): Containers = {
        val container1 = MySQLContainer.Def().start ()
        val container2 = PostgreSQLContainer.Def().start ()
        container1 and container2
      }
      
      it should "test" in withContainers { case mysqlContainer and pgContainer =>
        assert(mysqlContainer.jdbcUrl.nonEmpty && pgContainer.jdbcUrl.nonEmpty)
      }
      
    }


// First of all, you need to declare, which containers you want to use
override type Containers = MySQLContainer and PostgreSQLContainer

// After that, you need to describe, how you want to start them,
// In this method you can use any intermediate logic.
// You can pass parameters between containers, for example.
override def startContainers(): Containers = {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. is it possible to start the containers under the hood? The order of start can be inherited from the order of returned definitions
  2. is it possible to add default behavior for cases with no additional configuration (like this one in the doc)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to find a way to do this and I came to the conclusion that there is no need for this default behavior. First of all, you need to pass parameters inside the container's definitions. So, anyway, you need to call ContainerDef constructor. Where will you do this? Even if you do this inside a test body (like in the classic API) you will need the same amount of code.

Let's think about it. Imagine, that we have some default way to start containers in a defined order. I can imagine API like that:

class ExampleSpec extends FlatSpec with TestContainersForAll {

  val containers = MySQLContainer.Def(paramsHere) and PostgreSQLContainer.Def(paramsHere)
  
  // Tests here
}

And it looks exactly like the current API! With all its advantages and disadvantages. The main problem with it — how will you express dependent containers? Looks like you need a code block with the initialization logic:

class ExampleSpec extends FlatSpec with TestContainersForAll {

  val containers = {
    val c1 = Container1.Def()
    val c2 = Container2.Def(c1.someParam)
    c1 and c2
  }
  
  // Tests here
}

But, this code will not work, because c1.someParam will not be accessible, because you can get some parameters only from started containers. So, you need to start them:

class ExampleSpec extends FlatSpec with TestContainersForAll {

  val containers = {
    val c1 = Container1.Def().start()
    val c2 = Container2.Def(c1.someParam).start()
    c1 and c2
  }
  
  // Tests here
}

But you can't create a val inside your test body, because val is eager and will start to initialize with the test constructor. So, if you want to control containers initialization time, you need to make it def:

class ExampleSpec extends FlatSpec with TestContainersForAll {

  def startContainers = {
    // Initialization here
  }
  
  // Tests here
}

But, how will you provide started containers to the user? It looks like you need some function that will know about startup logic. Like withContainers:

class ExampleSpec extends FlatSpec with TestContainersForAll {

  def startContainers = {
    // Initialization here
  }
  
  it should "test" in withContainers { c1 and c2 =>
    // Test body
  }
}

withContainers should receive a function from containers type to a Unit. But what is containers type exactly? If you want to provide reusable traits like TestContainersForAll you need to abstract from this type somehow. And this is exactly what Containers type do. And initialization block — is the startContainers method. And eventually we end up with the same API:

class ExampleSpec extends FlatSpec with TestContainersForAll {

  override type Containers = Container1 and Container2

  override def startContainers(): Containers = {
    val c1 = Container1.Def().start()
    val c2 = Container2.Def(c1.someParam).start
    c1 and c2
  }
  
  it should "test" in withContainers { c1 and c2 =>
    // Test body
  }
}

Also, if you don't have dependent containers, you can write a bit less verbosely:

def startContainers() = Container1.Def().start() and Container2.Def().start()

Looks decent for me.

Actually, the reasoning above is the exact reasoning of this whole API. This is how I end up with this API.

val container1 = MySQLContainer.Def().start()
val container2 = PostgreSQLContainer.Def().start()
container1 and container2
}

// `withContainers` function supports multiple containers:
it should "test" in withContainers { case mysqlContainer and pgContainer =>
// Inside your test body you can do with your containers whatever you want to
assert(mysqlContainer.jdbcUrl.nonEmpty && pgContainer.jdbcUrl.nonEmpty)
}

}
```

Usage of `TestContainersForEach` is not different from the example above.

### `GenericContainer` usage

To create a custom container, which is not built-in in the library, you need to use `GenericContainer`.

For example, you want to create a custom nginx container:
```scala
class NginxContainer(port: Int, underlying: GenericContainer) extends GenericContainer(underlying) {
// you can add any methods or fields inside your container's body
def rootUrl: String = s"http://$containerIpAddress:${mappedPort(port)}/"
}
object NginxContainer {

// In the container definition you need to describe, how your container will be constructed:
case class Def(port: Int) extends GenericContainer.Def[NginxContainer](
new NginxContainer(port, GenericContainer(
dockerImage = "nginx:latest",
exposedPorts = Seq(port),
waitStrategy = Wait.forHttp("/")
))
)
}
```

### Migration from the classic API

1. If you have custom containers created with the `GenericContainer`, add `ContainerDef` in the companion like this:
```scala
object MyCustomContainer {
case class Def(/*constructor params here*/) extends GenericContainer.Def[MyCustomContainer](
new MyCustomContainer(/*constructor params here*/)
)
}
```
2. If you are using `ForEachTestContainer`:
1. If your test contains only one container, replace `ForEachTestContainer` with `TestContainerForEach`
2. If your test contains multiple containers, replace `ForEachTestContainer` with `TestContainersForEach`
3. If you are using `ForAllTestContainer`:
1. If your test contains only one container, replace `ForAllTestContainer` with `TestContainerForAll`
2. If your test contains multiple containers, replace `ForAllTestContainer` with `TestContainersForAll`
4. Fix all compilation errors using compiler messages and examples above.

If you have any questions or difficulties feel free to ask it in our [slack channel](https://testcontainers.slack.com/messages/CAFK4GL85).

## Release notes

* **0.34.0**
* Added new, experimental API and DSL.
The main motivation points are in the [pull request](https://github.com/testcontainers/testcontainers-scala/pull/78).
Old API remains the same, so all your old code will continue to work.
We will wait for the user's feedback about the new API.
If it will be positive, eventually this API may replace the current API.
You can find more information about the new API above.

* **0.33.0**
* TODO

* **0.32.0**
* TestContainers -> `1.12.1`
* SBT -> `1.3.0`
Expand Down
10 changes: 7 additions & 3 deletions core/src/main/scala/com/dimafeng/testcontainers/Container.scala
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,21 @@ package com.dimafeng.testcontainers

import java.util.function.Consumer

import com.dimafeng.testcontainers.lifecycle.Stoppable
import com.github.dockerjava.api.DockerClient
import com.github.dockerjava.api.command.{CreateContainerCmd, InspectContainerResponse}
import com.github.dockerjava.api.model.{Bind, Info, VolumesFrom}
import org.junit.runner.Description
import org.testcontainers.containers.output.OutputFrame
import org.testcontainers.containers.startupcheck.StartupCheckStrategy
import org.testcontainers.containers.traits.LinkableContainer
import org.testcontainers.containers.{FailureDetectingExternalResource, Network, TestContainerAccessor, GenericContainer => OTCGenericContainer}
import org.testcontainers.containers.{FailureDetectingExternalResource, Network, TestContainerAccessor, GenericContainer => JavaGenericContainer}
import org.testcontainers.lifecycle.Startable

import scala.collection.JavaConverters._
import scala.concurrent.{Future, blocking}

@deprecated("For internal usage only. Will be deleted.")
trait TestContainerProxy[T <: FailureDetectingExternalResource] extends Container {

@deprecated("Please use reflective methods from the wrapper and `configure` method for creation")
Expand All @@ -33,7 +35,9 @@ trait TestContainerProxy[T <: FailureDetectingExternalResource] extends Containe
override def failed(e: Throwable)(implicit description: Description): Unit = TestContainerAccessor.failed(e, description)
}

abstract class SingleContainer[T <: OTCGenericContainer[_]] extends TestContainerProxy[T] {
abstract class SingleContainer[T <: JavaGenericContainer[_]] extends TestContainerProxy[T] {

def underlyingUnsafeContainer: T = container

override def start(): Unit = container.start()

Expand Down Expand Up @@ -102,7 +106,7 @@ abstract class SingleContainer[T <: OTCGenericContainer[_]] extends TestContaine
}
}

trait Container extends Startable {
trait Container extends Startable with Stoppable {

@deprecated("Use `stop` instead")
def finished()(implicit description: Description): Unit = stop()
Expand Down
17 changes: 17 additions & 0 deletions core/src/main/scala/com/dimafeng/testcontainers/ContainerDef.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
package com.dimafeng.testcontainers

import com.dimafeng.testcontainers.lifecycle.Stoppable
import org.testcontainers.lifecycle.Startable

trait ContainerDef {

type Container <: Startable with Stoppable

protected def createContainer(): Container

def start(): Container = {
val container = createContainer()
container.start()
container
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import java.util.function.Consumer
import com.dimafeng.testcontainers.DockerComposeContainer.ComposeFile
import org.testcontainers.containers.output.OutputFrame
import org.testcontainers.containers.wait.strategy.{Wait, WaitStrategy}
import org.testcontainers.containers.{DockerComposeContainer => OTCDockerComposeContainer}
import org.testcontainers.containers.{DockerComposeContainer => JavaDockerComposeContainer}
import org.testcontainers.utility.Base58

import scala.collection.JavaConverters._
Expand Down Expand Up @@ -73,10 +73,10 @@ class DockerComposeContainer(composeFiles: ComposeFile,
env: Map[String, String] = Map.empty,
tailChildContainers: Boolean = false,
logConsumers: Seq[ServiceLogConsumer] = Seq.empty)
extends TestContainerProxy[OTCDockerComposeContainer[_]] {
extends TestContainerProxy[JavaDockerComposeContainer[_]] {

override val container: OTCDockerComposeContainer[_] = {
val container: OTCDockerComposeContainer[_] = new OTCDockerComposeContainer(identifier, composeFiles match {
override val container: JavaDockerComposeContainer[_] = {
val container: JavaDockerComposeContainer[_] = new JavaDockerComposeContainer(identifier, composeFiles match {
case ComposeFile(Left(f)) => util.Arrays.asList(f)
case ComposeFile(Right(files)) => files.asJava
})
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
package com.dimafeng.testcontainers

import org.testcontainers.containers.wait.strategy.WaitStrategy
import org.testcontainers.containers.{BindMode, FixedHostPortGenericContainer => OTCFixedHostPortGenericContainer}
import org.testcontainers.containers.{BindMode, FixedHostPortGenericContainer => JavaFixedHostPortGenericContainer}

class FixedHostPortGenericContainer(imageName: String,
exposedPorts: Seq[Int] = Seq(),
Expand All @@ -11,9 +11,9 @@ class FixedHostPortGenericContainer(imageName: String,
waitStrategy: Option[WaitStrategy] = None,
exposedHostPort: Int,
exposedContainerPort: Int
) extends SingleContainer[OTCFixedHostPortGenericContainer[_]] {
) extends SingleContainer[JavaFixedHostPortGenericContainer[_]] {

override implicit val container: OTCFixedHostPortGenericContainer[_] = new OTCFixedHostPortGenericContainer(imageName)
override implicit val container: JavaFixedHostPortGenericContainer[_] = new JavaFixedHostPortGenericContainer(imageName)

if (exposedPorts.nonEmpty) {
container.withExposedPorts(exposedPorts.map(int2Integer): _*)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,30 +4,41 @@ import java.util.concurrent.Future

import com.dimafeng.testcontainers.GenericContainer.DockerImage
import org.testcontainers.containers.wait.strategy.WaitStrategy
import org.testcontainers.containers.{BindMode, GenericContainer => OTCGenericContainer}

class GenericContainer(dockerImage: DockerImage,
exposedPorts: Seq[Int] = Seq(),
env: Map[String, String] = Map(),
command: Seq[String] = Seq(),
classpathResourceMapping: Seq[(String, String, BindMode)] = Seq(),
waitStrategy: Option[WaitStrategy] = None
) extends SingleContainer[OTCGenericContainer[_]] {

override implicit val container: OTCGenericContainer[_] = dockerImage match {
case DockerImage(Left(imageFromDockerfile)) => new OTCGenericContainer(imageFromDockerfile)
case DockerImage(Right(imageName)) => new OTCGenericContainer(imageName)
}
import org.testcontainers.containers.{BindMode, GenericContainer => JavaGenericContainer}

if (exposedPorts.nonEmpty) {
container.withExposedPorts(exposedPorts.map(int2Integer): _*)
}
env.foreach(Function.tupled(container.withEnv))
if (command.nonEmpty) {
container.withCommand(command: _*)
}
classpathResourceMapping.foreach(Function.tupled(container.withClasspathResourceMapping))
waitStrategy.foreach(container.waitingFor)
class GenericContainer(
override val underlyingUnsafeContainer: JavaGenericContainer[_]
) extends SingleContainer[JavaGenericContainer[_]] {

override implicit val container: JavaGenericContainer[_] = underlyingUnsafeContainer

def this(
dockerImage: DockerImage,
exposedPorts: Seq[Int] = Seq(),
env: Map[String, String] = Map(),
command: Seq[String] = Seq(),
classpathResourceMapping: Seq[(String, String, BindMode)] = Seq(),
waitStrategy: Option[WaitStrategy] = None
) = this({
val underlying: JavaGenericContainer[_] = dockerImage match {
case DockerImage(Left(imageFromDockerfile)) => new JavaGenericContainer(imageFromDockerfile)
case DockerImage(Right(imageName)) => new JavaGenericContainer(imageName)
}

if (exposedPorts.nonEmpty) {
underlying.withExposedPorts(exposedPorts.map(int2Integer): _*)
}
env.foreach(Function.tupled(underlying.withEnv))
if (command.nonEmpty) {
underlying.withCommand(command: _*)
}
classpathResourceMapping.foreach(Function.tupled(underlying.withClasspathResourceMapping))
waitStrategy.foreach(underlying.waitingFor)

underlying
})

def this(genericContainer: GenericContainer) = this(genericContainer.underlyingUnsafeContainer)
}

object GenericContainer {
Expand All @@ -48,4 +59,9 @@ object GenericContainer {
classpathResourceMapping: Seq[(String, String, BindMode)] = Seq(),
waitStrategy: WaitStrategy = null): GenericContainer =
new GenericContainer(dockerImage, exposedPorts, env, command, classpathResourceMapping, Option(waitStrategy))

abstract class Def[C <: GenericContainer](init: => C) extends ContainerDef {
override type Container = C
protected def createContainer(): C = init
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
package com.dimafeng.testcontainers.lifecycle

trait Stoppable extends AutoCloseable with Andable {

def stop(): Unit

override def close(): Unit = stop()
}

/**
* Gives you a possibility to write `container1 and container2`.
*
* Used for tests DSL.
*/
sealed trait Andable {

def stop(): Unit

def foreach(f: Stoppable => Unit): Unit = {
this match {
case and(head, tail) =>
head.foreach(f)
tail.foreach(f)

case stoppable: Stoppable =>
f(stoppable)
}
}

}
final case class and[A1 <: Andable, A2 <: Andable](head: A1, tail: A2) extends Andable {

/**
* Stopping all Andable elements in the reverse order
*/
override def stop(): Unit = {
tail.stop()
head.stop()
}
}

object Andable {
implicit class AndableOps[A <: Andable](val self: A) extends AnyVal {
def and[A2 <: Andable](that: A2): A and A2 = com.dimafeng.testcontainers.lifecycle.and(self, that)
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ package com.dimafeng.testcontainers

import org.mockito.MockitoAnnotations
import org.scalatest.{BeforeAndAfterAll, BeforeAndAfterEach, FlatSpec, Matchers}
import org.scalatest.mockito.MockitoSugar
import org.scalatestplus.mockito.MockitoSugar

abstract class BaseSpec[T: Manifest]
extends FlatSpec with Matchers with MockitoSugar with BeforeAndAfterEach with BeforeAndAfterAll {
Expand Down
Loading