Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MongoDB module #1961

Merged
merged 42 commits into from May 8, 2020
Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
3c07308
Implement MongoDbContainer
silaev Oct 7, 2019
a2c5c69
change oraclejdk8 with openjdk11, delete obsolete ITTest
silaev Oct 7, 2019
8ddd6b4
Small fixes
silaev Oct 7, 2019
1795235
Fix docs
silaev Oct 8, 2019
ee5e4fa
Merge branch 'master' of https://github.com/testcontainers/testcontai…
silaev Oct 9, 2019
e06749d
fix mongodb.md
silaev Oct 9, 2019
1a2b1dc
Merge branch 'master' of https://github.com/testcontainers/testcontai…
silaev Oct 11, 2019
8f68d71
Code review remarks adjusted
silaev Oct 14, 2019
58ab3f2
Merge branch 'master' of https://github.com/testcontainers/testcontai…
silaev Oct 14, 2019
957cd51
Small fix mongodb.md
silaev Oct 14, 2019
991a5f8
Small fix mongodb.md
silaev Oct 14, 2019
4d9f360
Merge branch 'master' into master
rnorth Jan 29, 2020
72d2b30
Merge branch 'master' into master
rnorth Feb 7, 2020
c0fe7ba
Merge branch 'master' into master
rnorth Feb 8, 2020
509a454
Merge branch 'master' of https://github.com/testcontainers/testcontai…
silaev Feb 26, 2020
d1d0367
Merge branch 'master' of https://github.com/testcontainers/testcontai…
silaev Mar 6, 2020
184d4e2
Fix container exit code check
silaev Mar 6, 2020
fc45144
Merge branch 'master' of https://github.com/testcontainers/testcontai…
silaev Mar 13, 2020
cfe5177
Adjust tests and build.gradle to follow Testcontainers module test co…
silaev Mar 13, 2020
a74d766
Add Javadoc explaining the use of LOCALHOST
silaev Mar 13, 2020
7620cb8
Refactor to use a default replica set configuration
silaev Mar 16, 2020
4ca7c7b
Merge branch 'master' of https://github.com/testcontainers/testcontai…
silaev Apr 17, 2020
aad0e76
Remove extra tests
silaev Apr 17, 2020
b92d690
Remove extra unit tests
silaev Apr 18, 2020
9a746df
Merge branch 'master' of https://github.com/testcontainers/testcontai…
silaev Apr 18, 2020
5c3182e
Merge branch 'master' of https://github.com/testcontainers/testcontai…
silaev Apr 20, 2020
5f3145b
Remove MockMaker
silaev Apr 24, 2020
2ea01ce
Merge branch 'master' of https://github.com/testcontainers/testcontai…
silaev Apr 24, 2020
8b727f0
Move initReplicaSet to containerIsStarted, set timeout delay to 100ms…
silaev Apr 24, 2020
e714b26
Make initReplicaSet private
silaev Apr 24, 2020
5b5531d
Fix docs
silaev Apr 24, 2020
d892006
Fix docs
silaev Apr 24, 2020
7adc9f8
Fix docs
silaev Apr 24, 2020
c136592
Make mongoDbContainer final in test
silaev Apr 24, 2020
a773cc5
Fix docs
silaev Apr 24, 2020
fb325d4
Fix docs, change MongoDb with MongoDB, remove logReplicaSetStatus
silaev Apr 24, 2020
649dae2
Reuse secondary constructor
silaev Apr 24, 2020
cd23ddb
Adjust heading
silaev Apr 24, 2020
47c39e7
Remove configureMongoDBContainer
silaev Apr 24, 2020
8e57d07
Merge branch 'master' into master
rnorth May 8, 2020
78ad147
Update docs/modules/databases/mongodb.md
rnorth May 8, 2020
7cdbad4
Merge branch 'master' into master
rnorth May 8, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
57 changes: 57 additions & 0 deletions docs/modules/databases/mongodb.md
@@ -0,0 +1,57 @@
# Mongo DB Module

!!! note
This module is INCUBATING. While it is ready for use and operational in the current version of Testcontainers, it is possible that it may receive breaking changes in the future. See [our contributing guidelines](/contributing/#incubating-modules) for more information on our incubating modules policy.

# Java8 MongoDbContainer for constructing a single node MongoDB replica set. To construct a multi-node MongoDB cluster, consider the [mongodb-replica-set project](https://github.com/silaev/mongodb-replica-set/)

## Usage example

The following example shows how to create a MongoDbContainer

<!--codeinclude-->
[Creating a MongoDB container](../../../modules/mongodb/src/test/java/org/testcontainers/containers/MongoDbContainerTest.java) inside_block:creatingMongoDbContainer
<!--/codeinclude-->

<!--codeinclude-->
[Starting a MongoDB container](../../../modules/mongodb/src/test/java/org/testcontainers/containers/MongoDbContainerTest.java) inside_block:startingMongoDbContainer
rnorth marked this conversation as resolved.
Show resolved Hide resolved
<!--/codeinclude-->

#### Motivation
Implement a reusable, cross-platform, simple to install solution that doesn't depend on
fixed ports to test MongoDB transactions.

#### General info
MongoDB starting form version 4 supports multi-document transactions only for a replica set.
rnorth marked this conversation as resolved.
Show resolved Hide resolved
For instance, to initialize a single node replica set on fixed ports via Docker, one has to do the following:

* Run a MongoDB container of version 4 and up specifying --replSet command
* Initialize a single replica set via executing a proper command
* Wait for the initialization to complete
* Provide a special url for a user to employ with a MongoDB driver without specifying replicaSet

As we can see, there is a lot of operations to execute and we even haven't touched a non-fixed port approach.
That's where the MongoDbContainer might come in handy.

## Adding this module to your project dependencies

Add the following dependency to your `pom.xml`/`build.gradle` file:

```groovy tab='Gradle'
testCompile "org.testcontainers:mongodb:{{latest_version}}"
```

```xml tab='Maven'
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>mongodb</artifactId>
<version>{{latest_version}}</version>
<scope>test</scope>
</dependency>
```

!!! hint
Adding this Testcontainers library JAR will not automatically add a database driver JAR to your project. You should ensure that your project also has a suitable database driver as a dependency

#### Copyright
Copyright (c) 2019 Konstantin Silaev <silaev256@gmail.com>
1 change: 1 addition & 0 deletions mkdocs.yml
Expand Up @@ -50,6 +50,7 @@ nav:
- modules/databases/dynalite.md
- modules/databases/influxdb.md
- modules/databases/mariadb.md
- modules/databases/mongodb.md
- modules/databases/mssqlserver.md
- modules/databases/mysql.md
- modules/databases/neo4j.md
Expand Down
7 changes: 7 additions & 0 deletions modules/mongodb/build.gradle
@@ -0,0 +1,7 @@
description = "Testcontainers :: MongoDB"

dependencies {
compile project(':testcontainers')

testCompile("org.mongodb:mongodb-driver-sync:4.0.2")
}
@@ -0,0 +1,142 @@
package org.testcontainers.containers;

import com.github.dockerjava.api.command.InspectContainerResponse;
import lombok.NonNull;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import org.testcontainers.containers.wait.strategy.Wait;

import java.io.IOException;

/**
rnorth marked this conversation as resolved.
Show resolved Hide resolved
* Constructs a single node MongoDB replica set for testing transactions.
* <p>To construct a multi-node MongoDB cluster, consider the <a href="https://github.com/silaev/mongodb-replica-set/">mongodb-replica-set project on GitHub</a>
* <p>Tested on a Mongo DB version 4.0.10+ (that is the default version if not specified).
*
* @author Konstantin Silaev on 9/30/2019
rnorth marked this conversation as resolved.
Show resolved Hide resolved
*/
@Slf4j
public class MongoDbContainer extends GenericContainer<MongoDbContainer> {
private static final int CONTAINER_EXIT_CODE_OK = 0;
private static final int MONGODB_INTERNAL_PORT = 27017;
private static final int AWAIT_INIT_REPLICA_SET_ATTEMPTS = 60;
private static final String MONGODB_VERSION_DEFAULT = "4.0.10";
private static final String MONGODB_DATABASE_NAME_DEFAULT = "test";

public MongoDbContainer() {
super("mongo:" + MONGODB_VERSION_DEFAULT);
rnorth marked this conversation as resolved.
Show resolved Hide resolved
configureMongoDbContainer();

}

public MongoDbContainer(@NonNull final String dockerImageName) {
super(dockerImageName);
configureMongoDbContainer();
}

private void configureMongoDbContainer() {
withExposedPorts(MONGODB_INTERNAL_PORT);
withCommand("--replSet", "docker-rs");
waitingFor(
Wait.forLogMessage(".*waiting for connections on port.*", 1)
);
}

public String getReplicaSetUrl() {
if (!isRunning()) {
rnorth marked this conversation as resolved.
Show resolved Hide resolved
throw new IllegalStateException("MongoDbContainer should be started first");
}
return String.format(
"mongodb://%s:%d/%s",
getContainerIpAddress(),
getMappedPort(MONGODB_INTERNAL_PORT),
MONGODB_DATABASE_NAME_DEFAULT
);
}

@Override
public void start() {
super.start();
logReplicaSetStatus();
rnorth marked this conversation as resolved.
Show resolved Hide resolved
}

@Override
protected void containerIsStarted(InspectContainerResponse containerInfo) {
initReplicaSet();
}

@SneakyThrows(value = {IOException.class, InterruptedException.class})
private void logReplicaSetStatus() {
log.debug(
"REPLICA SET STATUS:\n{}",
execInContainer(buildMongoEvalCommand("rs.status()")).getStdout()
);
}

private String[] buildMongoEvalCommand(final String command) {
return new String[]{"mongo", "--eval", command};
}

private void checkMongoNodeExitCode(final Container.ExecResult execResult) {
if (execResult.getExitCode() != CONTAINER_EXIT_CODE_OK) {
final String errorMessage = String.format("An error occurred: %s", execResult.getStdout());
log.error(errorMessage);
throw new ReplicaSetInitializationException(errorMessage);
}
}

private String buildMongoWaitCommand() {
return String.format(
"var attempt = 0; " +
"while" +
"(%s) " +
"{ " +
"if (attempt > %d) {quit(1);} " +
"print('%s ' + attempt); sleep(100); attempt++; " +
" }",
"db.runCommand( { isMaster: 1 } ).ismaster==false",
AWAIT_INIT_REPLICA_SET_ATTEMPTS,
"An attempt to await for a single node replica set initialization:"
);
}

private void checkMongoNodeExitCodeAfterWaiting(
final Container.ExecResult execResultWaitForMaster
) {
if (execResultWaitForMaster.getExitCode() != CONTAINER_EXIT_CODE_OK) {
final String errorMessage = String.format(
"A single node replica set was not initialized in a set timeout: %d attempts",
AWAIT_INIT_REPLICA_SET_ATTEMPTS
);
log.error(errorMessage);
throw new ReplicaSetInitializationException(errorMessage);
}
}

@SneakyThrows(value = {IOException.class, InterruptedException.class})
private void initReplicaSet() {
log.debug("Initializing a single node node replica set...");
final ExecResult execResultInitRs = execInContainer(
buildMongoEvalCommand("rs.initiate();")
);
log.debug(execResultInitRs.getStdout());
checkMongoNodeExitCode(execResultInitRs);

log.debug(
"Awaiting for a single node replica set initialization up to {} attempts",
AWAIT_INIT_REPLICA_SET_ATTEMPTS
);
final ExecResult execResultWaitForMaster = execInContainer(
buildMongoEvalCommand(buildMongoWaitCommand())
);
log.debug(execResultWaitForMaster.getStdout());

checkMongoNodeExitCodeAfterWaiting(execResultWaitForMaster);
}

public static class ReplicaSetInitializationException extends RuntimeException {
ReplicaSetInitializationException(final String errorMessage) {
super(errorMessage);
}
}
}
@@ -0,0 +1,74 @@
package org.testcontainers.containers;

import com.mongodb.ReadConcern;
import com.mongodb.ReadPreference;
import com.mongodb.TransactionOptions;
import com.mongodb.WriteConcern;
import com.mongodb.client.ClientSession;
import com.mongodb.client.MongoClient;
import com.mongodb.client.MongoClients;
import com.mongodb.client.MongoCollection;
import com.mongodb.client.TransactionBody;
import org.bson.Document;
import org.junit.Test;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;


public class MongoDbContainerTest {
/**
* Taken from <a href="https://docs.mongodb.com/manual/core/transactions/">https://docs.mongodb.com</a>
*/
@Test
public void shouldExecuteTransactions() {
try (
// creatingMongoDbContainer {
final MongoDbContainer mongoDbContainer = new MongoDbContainer()
// }
) {

// startingMongoDbContainer {
mongoDbContainer.start();
// }

final String mongoRsUrl = mongoDbContainer.getReplicaSetUrl();
assertNotNull(mongoRsUrl);
final MongoClient mongoSyncClient = MongoClients.create(mongoRsUrl);
mongoSyncClient.getDatabase("mydb1").getCollection("foo")
.withWriteConcern(WriteConcern.MAJORITY).insertOne(new Document("abc", 0));
mongoSyncClient.getDatabase("mydb2").getCollection("bar")
.withWriteConcern(WriteConcern.MAJORITY).insertOne(new Document("xyz", 0));

final ClientSession clientSession = mongoSyncClient.startSession();
final TransactionOptions txnOptions = TransactionOptions.builder()
.readPreference(ReadPreference.primary())
.readConcern(ReadConcern.LOCAL)
.writeConcern(WriteConcern.MAJORITY)
.build();

final String trxResult = "Inserted into collections in different databases";

TransactionBody<String> txnBody = () -> {
final MongoCollection<Document> coll1 =
mongoSyncClient.getDatabase("mydb1").getCollection("foo");
final MongoCollection<Document> coll2 =
mongoSyncClient.getDatabase("mydb2").getCollection("bar");

coll1.insertOne(clientSession, new Document("abc", 1));
coll2.insertOne(clientSession, new Document("xyz", 999));
return trxResult;
};

try {
final String trxResultActual = clientSession.withTransaction(txnBody, txnOptions);
assertEquals(trxResult, trxResultActual);
} catch (RuntimeException re) {
throw new IllegalStateException(re.getMessage(), re);
} finally {
clientSession.close();
mongoSyncClient.close();
}
}
}
}
16 changes: 16 additions & 0 deletions modules/mongodb/src/test/resources/logback-test.xml
@@ -0,0 +1,16 @@
<configuration>

<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<!-- encoders are assigned the type
ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
<encoder>
<pattern>%d{HH:mm:ss.SSS} %-5level %logger - %msg%n</pattern>
</encoder>
</appender>

<root level="INFO">
<appender-ref ref="STDOUT"/>
</root>

<logger name="org.testcontainers" level="DEBUG"/>
</configuration>