Skip to content
Lightweight MongoDB migration tool for Java
Java Shell
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.infra
community
misc
mongock-core
mongock-spring
.editorconfig
.gitignore
.travis.yml
CHANGELOG.md
CODEOWNERS
DEPENDENCY_UPGRADE_CONSIDERATIONS.md
LICENSE.md
README.md
pom.xml
release-perform.sh

README.md

Maven Central Build Status Bugs Vulnerabilities Hex.pm

LAST NEWS ‼️‼️💥💥

3.0.3.BETA is released. Take a look to our changelog to find out what is new.

Mongock: MongoDB version control tool for Java

mongock is a java MongoDB tool for tracking, managing and applying database schema changes.

The motivation of this project is to add some important and useful features, provide a different code design, which we believe is easier to maintain, extend and debug, applying what we believe are best practices, and specially providing a more fluent maintenance to apply collaborator's contributions. The concept is very similar to other db migration tools such as Liquibase or Flyway but without using XML/JSON/YML files.

mongock provides new approach for adding changes (change sets) based on Java classes and methods with appropriate annotations.

Table of contents

Why Mongock

There are several good reasons to use Mongock in your project. Here we give you some of them:

  • Solid solution which really works.
  • Works well with sharded collections: Unlike other similar projects using javascript, which requires db.eval(). Documentation.
  • Distributed solution with solid locking mechanism.
  • We are very responsive, aiming for 24-hours-response for new issues and 48 hours for reviews(Notice this is not a commitment, but so far so good).
  • Well maintained and regularly updated.
  • Used by several tech companies in different industries.
  • Can be used together with most, if not all, frameworks.
  • Provides great integration for Spring, allowing you to inject any dependency you want to your changelog method.

Sample projects

In here you can find some sample projects that show you how to use Mongock.

Contributing

If you would like to contribute to Mongock project, please read how to contribute for details on our collaboration process and standards.

Add a dependency

Mongock can be used standalone or with Spring. The mongock-core dependency is always required, and mongock-spring can also be added. Using mongock-spring is not currently supported.

With Maven

<!-- To use standalone (i.e., w/o Spring ) -->
<dependency>
  <groupId>com.github.cloudyrock.mongock</groupId>
  <artifactId>mongock-core</artifactId>
  <version>3.0.3.BETA</version>
</dependency>

<!-- Add to use with Spring-->
<dependency>
  <groupId>com.github.cloudyrock.mongock</groupId>
  <artifactId>mongock-spring</artifactId>
  <version>3.0.3.BETA</version>
</dependency>

With Gradle

compile 'org.javassist:javassist:3.18.2-GA'          // workround for ${javassist.version} placeholder issue*
compile 'com.github.cloudyrock.mongock:mongock-core:3.0.3.BETA'    // standalone
compile 'com.github.cloudyrock.mongock:mongock-spring:3.0.3.BETA'  // with Spring (in addition to mongock-core)

Usage with Spring...Mongock as a Bean

You need to instantiate mongock object and provide some configuration. If you use Spring, mongock can be instantiated as a singleton bean in the Spring context. In this case the migration process will be executed automatically on startup.

@Bean
public SpringMongock mongock() {
  MongoClient mongoclient = new MongoClient(new MongoClientURI("yourDbName", yourMongoClientBuilder));
  return new SpringMongockBuilder(mongoclient, "yourDbName", "com.package.to.be.scanned.for.changesets")
      .setLockQuickConfig()
      .build();
}

Usage with SpringBoot...When you need to inject your own dependencies

The main benefit of using SpringBoot integration is that it provides a totally flexible way to inject dependencies, so you can inject any object to your change logs by using SpringBoot ApplicationContext.

In order to use this feature you need to instantiate the SpringBoot mongock class and provide the required configuration. Mongock will run as an ApplicationRunner within SpringBoot. In terms of execution, it will be be very similar to the standard Spring implementation. The key difference is that ApplicationRunner beans run after (as opposed to during) the context is fully initialized.

Note: Using this implementation means you need all the dependencies in your changelogs(parameters in methods annotated with @ChangeSet) declared as Spring beans.

Note: The dependencies injected by the ApplicationContext (other than MongoTemplate, MongoDatabase and DB) won't be covered by the lock. This means that if you are accessing to Mongo through a different mechanism to the ones mentioned, the lock synchronization is not guaranteed as Mongock only ensures synchronization when Mongo is accessed via either MongoTemplate, MongoDatabase or DB. For more information, please consult the lock section

@Bean
public SpringBootMongock mongock(ApplicationContext springContext, MongoClient mongoClient) {
  return new SpringBootMongockBuilder(mongoClient, "yourDbName", "com.package.to.be.scanned.for.changesets")
      .setApplicationContext(springContext) 
      .setLockQuickConfig()
      .build();
}

Standalone usage

Using mongock standalone.

  MongoClient mongoclient = new MongoClient(new MongoClientURI("yourDbName", yourMongoClientBuilder));
  Mongock runner=  new MongockBuilder(mongoclient, "yourDbName", "com.package.to.be.scanned.for.changesets")
      .setLockQuickConfig()
      .build();
  runner.execute();         //  ------> starts migration changesets

Above examples provide minimal configuration. The various Mongock builders provide some other possibilities (setters) to make the tool more flexible:

builder.setChangelogCollectionName(logColName);   // default is dbchangelog, collection with applied change sets
builder.setLockCollectionName(lockColName);       // default is mongocklock, collection used during migration process
builder.setEnabled(shouldBeEnabled);              // default is true, migration won't start if set to false

More about URI

Creating change logs

ChangeLog contains bunch of ChangeSets. ChangeSet is a single task (set of instructions made on a database). In other words ChangeLog is a class annotated with @ChangeLog and containing methods annotated with @ChangeSet.

package com.example.yourapp.changelogs;

@ChangeLog
public class DatabaseChangelog {
  
  @ChangeSet(order = "001", id = "someChangeId", author = "testAuthor")
  public void importantWorkToDo(DB db){
     // task implementation
  }


}

@ChangeLog

Class with change sets must be annotated by @ChangeLog. There can be more than one change log class but in that case order argument should be provided:

@ChangeLog(order = "001")
public class DatabaseChangelog {
  //...
}

ChangeLogs are sorted alphabetically by order argument and changesets are applied due to this order.

@ChangeSet

Method annotated by @ChangeSet is taken and applied to the database. History of applied change sets is stored in a collection called dbchangelog (by default) in your MongoDB

Annotation parameters:

order - string for sorting change sets in one changelog. Sorting in alphabetical order, ascending. It can be a number, a date etc.

id - name of a change set, must be unique for all change logs in a database

author - author of a change set

runAlways - [optional, default: false] changeset will always be executed but only first execution event will be stored in dbchangelog collection

version - [optional, default: "0"] defines a version on which this changeset is relate to. E.g. "0.1" means, this changeset should be applied to schema version 0.1 of your MongoDB. See Defining ChangeSet methods with versions for more information.

Defining ChangeSet methods

Method annotated by @ChangeSet can have one of the following definition:

@ChangeSet(order = "001", id = "someChangeWithoutArgs", author = "testAuthor")
public void someChange1() {
   // method without arguments can do some non-db changes
}

@ChangeSet(order = "002", id = "someChangeWithMongoDatabase", author = "testAuthor")
public void someChange2(MongoDatabase db) {
  // type: com.mongodb.client.MongoDatabase : original MongoDB driver v. 3.x, operations allowed by driver are possible
  // example: 
  MongoCollection<Document> mycollection = db.getCollection("mycollection");
  Document doc = new Document("testName", "example").append("test", "1");
  mycollection.insertOne(doc);
}

@ChangeSet(order = "003", id = "someChangeWithDb", author = "testAuthor")
public void someChange3(DB db) {
  // This is deprecated in mongo-java-driver 3.x, use MongoDatabase instead
  // type: com.mongodb.DB : original MongoDB driver v. 2.x, operations allowed by driver are possible
  // example: 
  DBCollection mycollection = db.getCollection("mycollection");
  BasicDBObject doc = new BasicDBObject().append("test", "1");
  mycollection .insert(doc);
}

@ChangeSet(order = "005", id = "someChangeWithSpringDataTemplate", author = "testAuthor")
public void someChange5(MongoTemplate mongoTemplate) {
  // type: org.springframework.data.mongodb.core.MongoTemplate
  // Spring Data integration allows using MongoTemplate in the ChangeSet
  // example:
  mongoTemplate.save(myEntity);
}

@ChangeSet(order = "006", id = "someChangeWithSpringDataTemplate", author = "testAuthor")
public void someChange6(MongoTemplate mongoTemplate, Environment environment) {
  // type: org.springframework.data.mongodb.core.MongoTemplate
  // type: org.springframework.core.env.Environment
  // Spring Data integration allows using MongoTemplate and Environment in the ChangeSet
}

Defining ChangeSet methods with versions(BETA...we accept suggestions)

Method annotated by @ChangeSet have also the possibility to contain a version. This is a useful feature from a consultancy point of view. The more descriptive scenario is where a software provider has several customers who he provides his software to. The clients may be using different versions of the software at the same time. So when he install the product in a customer, the changesets need to be applied depending on the product version. With this solution, he can tag every changeset with his product version and will tell mongock which version range to apply.

@ChangeSet(order = "001", id = "someChangeToVersionOne", author = "testAuthor", version = "1")
public void someChange1(MongoDatabase db) {
}

@ChangeSet(order = "002", id = "someChangeToVersionOneDotOne", author = "testAuthor", version = "1.1")
public void someChange2(MongoDatabase db) {
}

@ChangeSet(order = "003", id = "someChangeToVersionTwoDotFiveDotOne", author = "testAuthor", version = "2.5.1")
public void someChange3(MongoDatabase db) {
}

@ChangeSet(order = "004", id = "someChangeToVersionTwoDotFiveDotFive", author = "testAuthor", version = "2.5.5")
public void someChange5(MongoDatabase db) {
}

@ChangeSet(order = "005", id = "someChangeToVersionTwoDotSix", author = "testAuthor", version = "2.6")
public void someChange6(MongoDatabase db) {
}

With specifying versions you are able to upgrade to specific versions:

  MongoClient mongoclient = new MongoClient(new MongoClientURI("yourDbName", yourMongoClientBuilder));
  Mongock runner=  new MongockBuilder(mongoclient, "yourDbName", "com.package.to.be.scanned.for.changesets")
      .setLockQuickConfig()
      .setStartVersion("1")
      .setEndVersion("2.5.5")
      .build();
  runner.execute();         //  ------> starts migration changesets from version 1 to 2.5.5

This example will execute ChangeSet 1, 2 and 3, because the specified version in the changeset should be greater equals the startVersion and lower than endVersion.

Injecting custom dependencies to change logs

Right now this is possible by using SpringBoot Application Context. See SpringBoot set up for more information.

Using Spring profiles

mongock accepts Spring's org.springframework.context.annotation.Profile annotation. If a change log or change set class is annotated with @Profile, then it is activated for current application profiles.

Example 1: annotated change set will be invoked for a dev profile

@Profile("dev")
@ChangeSet(author = "testuser", id = "myDevChangest", order = "01")
public void devEnvOnly(DB db){
  // ...
}

Example 2: all change sets in a changelog will be invoked for a test profile

@ChangeLog(order = "1")
@Profile("test")
public class ChangelogForTestEnv{
  @ChangeSet(author = "testuser", id = "myTestChangest", order = "01")
  public void testingEnvOnly(DB db){
    // ...
  } 
}

Enabling @Profile annotation (option)

To enable the @Profile integration, please inject org.springframework.core.env.Environment to your runner.

@Bean 
@Autowired
public SpringMongock mongock(Environment environment) {
  SpringMongock runner = new SpringMongockBuilder(mongoclient, "yourDbName", "com.package.to.be.scanned.for.changesets")
      .setSpringEnvironment(environment)
      .setLockQuickConfig()
      .build();

  //... etc
}

Configuring Lock

In order to execute the changelogs, mongock needs to manage the lock to ensure only one instance executes a changelog at a time. By default the lock is reserved 24 hours and, in case the lock is held by another mongock instance, will ignore the execution and no exception will be sent, unless the parameter throwExceptionIfCannotObtainLock is set to true.

There are 3 parameters to configure:

lockAcquiredForMinutes - Number of minutes mongock will acquire the lock for. It will refresh the lock when is close to be expired anyway.

maxTries - Max tries when the lock is held by another mongock instance.

maxWaitingForLockMinutes - Max minutes mongock will wait for the lock in every try.

To configure these parameters there are two methods: setLockConfig and setLockConfig and setLockQuickConfig. Both will set the parameter throwExceptionIfCannotObtainLock to true.

@Bean @Autowired
public Mongock mongock(Environment environment) {
  Mongock runner = new mongock(uri);
  runner.setLockConfig(5, 6, 3);
}

or quick config with 3 minutes for lockAcquiredFor, 3 max tries and 4 minutes for maxWaitingForLock

 @Bean @Autowired
 public Mongock mongock(Environment environment) {
   Mongock runner = new mongock(uri);
   runner.setLockQuickConfig();
 }

Known issues

Mongo java driver conflicts

mongock depends on mongo-java-driver. If your application has mongo-java-driver dependency too, there could be library conflicts in some cases.

Exception:

com.mongodb.WriteConcernException: { "serverUsed" : "localhost" , 
"err" : "invalid ns to index" , "code" : 10096 , "n" : 0 , 
"connectionId" : 955 , "ok" : 1.0}

Workaround:

You can exclude mongo-java-driver from mongock and use your dependency only. Maven and gradle examples below:

<!-- pom.xml -->
<dependency>
    <groupId>org.mongodb</groupId>
    <artifactId>mongo-java-driver</artifactId>
    <version>3.4.0</version>
</dependency>

<dependency>
  <groupId>com.github.cloudyrock.mongock</groupId>
  <artifactId>mongock-core</artifactId>
  <version>3.0.3.BETA</version>
  <exclusions>
    <exclusion>
      <groupId>org.mongodb</groupId>
      <artifactId>mongo-java-driver</artifactId>
    </exclusion>
  </exclusions>
</dependency>
    // build.gradle
    compile "org.mongodb:mongo-java-driver:3.4.0"
    compile("com.github.cloudyrock.mongock:mongock:3.0.3.BETA") {
        exclude group: 'org.mongodb', module: 'mongo-java-driver'
    }

Mongo transaction limitations

Due to Mongo limitations, there is no way to provide atomicity at ChangelogSet level. So a Changelog could need more than one execution to be finished, as any interruption could happen, leaving the changelog in a inconsistent state. If that happens, the next time mongock is executed it will try to finish the changelog execution, but it could already be half executed.

For this reason, the developer in charge of the changelog's design, should make sure that:

  • Changelog is idempotent: As changelog can be interrupted at any time, it will need to be executed again.
  • Changelog is Backward compatible (If high availability is required): While the migration process is taking place, the old version of the software is still running. During this time could happen(and probably will) that the old version of the software is dealing with the new version of the data. Could even happen that the data is a mix between old and new version. This means the software must still work regardless of the status of the database. In case the developer is aware of this and still decides to provide a non-backward-compatible changeSet, he should know it's a detriment to high availability.
  • Changelog reduces its execution time in every iteration: This is harder to explain. As said, a changelog can be interrupted at any time. This means an specific changelog needs to be re-run. In the undesired scenario where the changelog's execution time is grater than the interruption time(could be Kubernetes initial delay), that changelog won't be ever finished. So the changelog needs to be developed in such a way that every iteration reduces its execution time, so eventually, after some iterations, the changelog finished.
  • Changelog's execution time is shorter than interruption time: In case the previous condition cannot be ensured, could be enough if the changelog's execution time is shorter than the interruption time. This is not ideal as the execution time depends on the machine, but in most case could be enough.

Code of conduct

Please read the code of conduct for details on our code of conduct.

LICENSE

Mongock propject is licensed under the Apache License Version 2.0. See the LICENSE file for details

You can’t perform that action at this time.