MongoConverter support in Spring Modules

[savvadem]

Hi,
I have some problems with mongock.
It seems that MongoTemplateDecoratorImpl doesn't catch

    @Bean
    public MongoCustomConversions mongoCustomConversions() {
        return new MongoCustomConversions(Arrays.asList(
                new BigDecimalDecimal128Converter(),
                new Decimal128BigDecimalConverter()
        ));

    }

and in the end i receive this exception:

Caused by: com.github.cloudyrock.mongock.MongockException: No converter found capable of converting from type [org.bson.types.Decimal128] to type [java.math.BigDecimal]
	at com.github.cloudyrock.mongock.Mongock.executeMigration(Mongock.java:119) ~[mongock-core-3.0.2.BETA.jar:na]
	at com.github.cloudyrock.mongock.Mongock.execute(Mongock.java:62) ~[mongock-core-3.0.2.BETA.jar:na]
	at com.github.cloudyrock.mongock.SpringBootMongock.run(SpringBootMongock.java:40) ~[mongock-spring-3.0.2.BETA.jar:na]
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:809) [spring-boot-2.0.5.RELEASE.jar:2.0.5.RELEASE]
	... 5 common frames omitted
Caused by: java.lang.reflect.InvocationTargetException: null
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_144]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_144]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_144]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_144]
	at com.github.cloudyrock.mongock.SpringBootMongock.executeChangeSetMethod(SpringBootMongock.java:66) ~[mongock-spring-3.0.2.BETA.jar:na]
	at com.github.cloudyrock.mongock.Mongock.executeIfNewOrRunAlways(Mongock.java:128) ~[mongock-core-3.0.2.BETA.jar:na]
	at com.github.cloudyrock.mongock.Mongock.executeMigration(Mongock.java:112) ~[mongock-core-3.0.2.BETA.jar:na]
	... 8 common frames omitted
Caused by: org.springframework.core.convert.ConverterNotFoundException: No converter found capable of converting from type [org.bson.types.Decimal128] to type [java.math.BigDecimal]
	at org.springframework.core.convert.support.GenericConversionService.handleConverterNotFound(GenericConversionService.java:321) ~[spring-core-5.0.9.RELEASE.jar:5.0.9.RELEASE]
	at org.springframework.core.convert.support.GenericConversionService.convert(GenericConversionService.java:194) ~[spring-core-5.0.9.RELEASE.jar:5.0.9.RELEASE]
	at org.springframework.core.convert.support.GenericConversionService.convert(GenericConversionService.java:174) ~[spring-core-5.0.9.RELEASE.jar:5.0.9.RELEASE]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.getPotentiallyConvertedSimpleRead(MappingMongoConverter.java:908) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.readValue(MappingMongoConverter.java:1412) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter$MongoDbPropertyValueProvider.getPropertyValue(MappingMongoConverter.java:1355) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.readProperties(MappingMongoConverter.java:336) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:292) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:245) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:194) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:190) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:78) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at org.springframework.data.mongodb.core.MongoTemplate$ReadDocumentCallback.doWith(MongoTemplate.java:2800) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at org.springframework.data.mongodb.core.MongoTemplate.executeFindMultiInternal(MongoTemplate.java:2463) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at org.springframework.data.mongodb.core.MongoTemplate.findAll(MongoTemplate.java:1650) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at com.github.cloudyrock.mongock.decorator.impl.MongoTemplateDecoratorImpl.lambda$findAll$61(MongoTemplateDecoratorImpl.java:420) ~[mongock-spring-3.0.2.BETA.jar:na]
	at com.github.cloudyrock.mongock.decorator.util.MethodInvokerImpl.invoke(MethodInvokerImpl.java:18) ~[mongock-core-3.0.2.BETA.jar:na]
	at com.github.cloudyrock.mongock.decorator.impl.MongoTemplateDecoratorImpl.findAll(MongoTemplateDecoratorImpl.java:420) ~[mongock-spring-3.0.2.BETA.jar:na]
	at org.springframework.data.mongodb.core.MongoTemplate.findAll(MongoTemplate.java:1645) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at com.github.cloudyrock.mongock.decorator.impl.MongoTemplateDecoratorImpl.lambda$findAll$60(MongoTemplateDecoratorImpl.java:415) ~[mongock-spring-3.0.2.BETA.jar:na]
	at com.github.cloudyrock.mongock.decorator.util.MethodInvokerImpl.invoke(MethodInvokerImpl.java:18) ~[mongock-core-3.0.2.BETA.jar:na]
	at com.github.cloudyrock.mongock.decorator.impl.MongoTemplateDecoratorImpl.findAll(MongoTemplateDecoratorImpl.java:415) ~[mongock-spring-3.0.2.BETA.jar:na]

dependency version:
compile 'com.github.cloudyrock.mongock:mongock-spring:3.0.2.BETA'
Spring Boot 2.0.5.RELEASE

[savvadem]

As workaround, i used aggregation and bson.Document, so i didn't need converters.

[dieppa]

thanks, we'll update you on this.

[dieppa]

Hi, we are quite busy right now. is this fix quite important for you or can wait?

[savvadem]

Sure, i can wait, we are in pre-prod stage right now.

[savvadem]

Any updates here?

[dieppa]

Hi @savvadem , I haven't able to look at this. As you had a workaround we have prioritise other things, like bugs, etc. Version 3 has been a major change internally, that's the reason we're still in BETA.

I cannot tell you this will be fixed this week, but I will try to have a look to at least understand what we are talking about.

Thanks

[Ovaro]

I am having the same issue after upgrading to the non-BETA version 3.2.3 from 2.0.2. I am using Running with Spring Boot v2.1.11.RELEASE, Spring v5.1.12.RELEASE.

Exception:

Caused by: com.github.cloudyrock.mongock.MongockException: Can't find a codec for class java.time.ZonedDateTime.
at com.github.cloudyrock.mongock.Mongock.executeMigration(Mongock.java:118)
at com.github.cloudyrock.mongock.Mongock.execute(Mongock.java:62)
at com.github.cloudyrock.mongock.SpringBootMongock.run(SpringBootMongock.java:35)
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:772)
... 8 common frames omitted
Caused by: java.lang.reflect.InvocationTargetException: null
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.github.cloudyrock.mongock.SpringBootMongock.executeChangeSetMethod(SpringBootMongock.java:60)
at com.github.cloudyrock.mongock.Mongock.executeIfNewOrRunAlways(Mongock.java:127)
at com.github.cloudyrock.mongock.Mongock.executeMigration(Mongock.java:111)
... 11 common frames omitted
Caused by: org.bson.codecs.configuration.CodecConfigurationException: Can't find a codec for class java.time.ZonedDateTime.
at org.bson.codecs.configuration.CodecCache.getOrThrow(CodecCache.java:46)
at org.bson.codecs.configuration.ProvidersCodecRegistry.get(ProvidersCodecRegistry.java:63)
at org.bson.codecs.configuration.ChildCodecRegistry.get(ChildCodecRegistry.java:51)
at org.bson.codecs.DocumentCodec.writeValue(DocumentCodec.java:184)
at org.bson.codecs.DocumentCodec.writeMap(DocumentCodec.java:199)
at org.bson.codecs.DocumentCodec.encode(DocumentCodec.java:141)
at org.bson.codecs.DocumentCodec.encode(DocumentCodec.java:45)
at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:63)
at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:29)
at com.mongodb.operation.BulkWriteBatch$WriteRequestEncoder.encode(BulkWriteBatch.java:387)
at com.mongodb.operation.BulkWriteBatch$WriteRequestEncoder.encode(BulkWriteBatch.java:377)
at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:63)
at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:29)
at com.mongodb.internal.connection.BsonWriterHelper.writeDocument(BsonWriterHelper.java:75)
at com.mongodb.internal.connection.BsonWriterHelper.writePayload(BsonWriterHelper.java:59)
at com.mongodb.internal.connection.CommandMessage.encodeMessageBodyWithMetadata(CommandMessage.java:143)
at com.mongodb.internal.connection.RequestMessage.encode(RequestMessage.java:138)
at com.mongodb.internal.connection.CommandMessage.encode(CommandMessage.java:57)
at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:244)
at com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:99)
at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:444)
at com.mongodb.internal.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:72)
at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:200)
at com.mongodb.internal.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:269)
at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:131)
at com.mongodb.operation.MixedBulkWriteOperation.executeCommand(MixedBulkWriteOperation.java:418)
at com.mongodb.operation.MixedBulkWriteOperation.executeBulkWriteBatch(MixedBulkWriteOperation.java:256)
at com.mongodb.operation.MixedBulkWriteOperation.access$700(MixedBulkWriteOperation.java:67)
at com.mongodb.operation.MixedBulkWriteOperation$1.call(MixedBulkWriteOperation.java:200)
at com.mongodb.operation.MixedBulkWriteOperation$1.call(MixedBulkWriteOperation.java:191)
at com.mongodb.operation.OperationHelper.withReleasableConnection(OperationHelper.java:424)
at com.mongodb.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:191)
at com.mongodb.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:67)
at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:193)
at com.mongodb.client.internal.MongoCollectionImpl.executeSingleWriteRequest(MongoCollectionImpl.java:960)
at com.mongodb.client.internal.MongoCollectionImpl.executeInsertOne(MongoCollectionImpl.java:494)
at com.mongodb.client.internal.MongoCollectionImpl.insertOne(MongoCollectionImpl.java:478)
at com.mongodb.client.internal.MongoCollectionImpl.insertOne(MongoCollectionImpl.java:472)
at com.github.cloudyrock.mongock.decorator.MongoCollectionDecorator.lambda$insertOne$15(MongoCollectionDecorator.java:158)
at com.github.cloudyrock.mongock.decorator.util.MethodInvokerImpl.invoke(MethodInvokerImpl.java:24)
at com.github.cloudyrock.mongock.decorator.MongoCollectionDecorator.insertOne(MongoCollectionDecorator.java:158)
at com.github.cloudyrock.mongock.decorator.MongoCollectionDecorator.lambda$insertOne$15(MongoCollectionDecorator.java:158)
at com.github.cloudyrock.mongock.decorator.util.MethodInvokerImpl.invoke(MethodInvokerImpl.java:24)
at com.github.cloudyrock.mongock.decorator.MongoCollectionDecorator.insertOne(MongoCollectionDecorator.java:158)
at com.github.cloudyrock.mongock.decorator.MongoCollectionDecorator.lambda$insertOne$15(MongoCollectionDecorator.java:158)
at com.github.cloudyrock.mongock.decorator.util.MethodInvokerImpl.invoke(MethodInvokerImpl.java:24)
at com.github.cloudyrock.mongock.decorator.MongoCollectionDecorator.insertOne(MongoCollectionDecorator.java:158)
at org.springframework.data.mongodb.core.MongoTemplate$7.doInCollection(MongoTemplate.java:1498)
at org.springframework.data.mongodb.core.MongoTemplate.execute(MongoTemplate.java:544)
at com.github.cloudyrock.mongock.decorator.impl.MongoTemplateDecoratorImpl.lambda$execute$8(MongoTemplateDecoratorImpl.java:145)
at com.github.cloudyrock.mongock.decorator.util.MethodInvokerImpl.invoke(MethodInvokerImpl.java:18)
at com.github.cloudyrock.mongock.decorator.impl.MongoTemplateDecoratorImpl.execute(MongoTemplateDecoratorImpl.java:145)
at org.springframework.data.mongodb.core.MongoTemplate.saveDocument(MongoTemplate.java:1488)
at org.springframework.data.mongodb.core.MongoTemplate.doSave(MongoTemplate.java:1423)
at org.springframework.data.mongodb.core.MongoTemplate.save(MongoTemplate.java:1373)
at org.springframework.data.mongodb.core.MongoTemplate.save(MongoTemplate.java:1359)
at io.ovaro.config.dbmigrations.InitialSetupMigration.job1UnassignedFixedManual(InitialSetupMigration.java:2646)
at io.ovaro.config.dbmigrations.InitialSetupMigration.addJobs(InitialSetupMigration.java:2338)
at io.ovaro.config.dbmigrations.InitialSetupMigration.addJobs(InitialSetupMigration.java:2328)
... 18 common frames omitted

[dieppa]

Sorry , I didn't update on this. I wasn't able to reproduce this, and we was distracted by some other more urgent matters. We'll look at this today. However, it would be nice if you can provide an example app that I can checkout out and reproduce easily.

Thanks @Ovaro

[Ovaro]

Hi @dieppa - this is on a Jhipster based app where I have moved to monogock rather than mongobee.

@Configuration @EnableMongoRepositories(basePackages = "xxx.repository", includeFilters = @Filter(type = FilterType.ASSIGNABLE_TYPE, value = MongoRepository.class)) @Profile("!" + JHipsterConstants.SPRING_PROFILE_CLOUD) @Import(value = MongoAutoConfiguration.class) @EnableMongoAuditing(auditorAwareRef = "springSecurityAuditorAware") public class DatabaseConfiguration {

private final Logger log = LoggerFactory.getLogger(DatabaseConfiguration.class);

@Bean
public ValidatingMongoEventListener validatingMongoEventListener() {
    return new ValidatingMongoEventListener(validator());
}

@Bean
public LocalValidatorFactoryBean validator() {
    return new LocalValidatorFactoryBean();
}

@Bean
public MongoCustomConversions customConversions() {
    log.debug("Adding custom conversions ZZZ");
    List<Converter<?, ?>> converters = new ArrayList<>();
    converters.add(DateToZonedDateTimeConverter.INSTANCE);
    converters.add(ZonedDateTimeToDateConverter.INSTANCE);
    return new MongoCustomConversions(converters);
}

@Bean
public SpringBootMongock mongock(ApplicationContext springContext, MongoClient mongoClient, MongoTemplate mongoTemplate, MongoProperties mongoProperties) {
    return new SpringBootMongockBuilder(mongoClient, mongoProperties.getMongoClientDatabase(), "xxxx.config.dbmigrations")
        .setApplicationContext(springContext)
        .setLockQuickConfig()
        .build();
}

// The following converter definitions to fix org.bson.codecs.configuration.CodecConfigurationException: Can't find a codec for class java.time.ZonedDateTime

@WritingConverter
public enum ZonedDateTimeToDateConverter implements Converter<ZonedDateTime, Date> {
    INSTANCE;

    @Override
    public Date convert(ZonedDateTime source) {
        return source == null ? null : Date.from(source.toInstant());
    }
}

@ReadingConverter
public enum DateToZonedDateTimeConverter implements Converter<Date, ZonedDateTime> {
    INSTANCE;

    @Override
    public ZonedDateTime convert(Date source) {
        return source == null ? null : ZonedDateTime.ofInstant(source.toInstant(), ZoneId.systemDefault());
    }
}

}

And the example of a class that doesn't work:

@document(collection = "job")
public class Job {
@id
private String id;
private String title;
@NotNull
private String state;

@NotNull
private ZonedDateTime lastUpdated;
@NotNull
private ZonedDateTime posted;

....
}

This all worked in version 2.0.2 but has stopped in version 3 and above. Mongock is being created in exactly the same way so have the converters registered (those debug log lines stating the converter is registered occurs way before the errors).

[dieppa]

Hello, that's probably because we migrate from a proxy approach to control the lock to implement decorators.

Anyway, we have been able to reproduce it and doesn't seem that will be hard to fix. It will be fixed by Thursday evening.

Cheers

[Ovaro]

@dieppa Awesome! Great to hear it wasn't a big problem. I didn't look into the cause as I saw this open issue. Really appreciated. Cheers.

[dieppa]

@Ovaro, I believe that if you use MongoTemplate, instead of MongoClient, it works fine. Can you check, please?

Just notice that currently there is a bug using MongoTemplate, which affects for example to fields with annotated with @indexed. However this is fixed, but not released yet. Will be released before Thursday, together with any fix this issue require, if it's the case.

[Ovaro]

@dieppa Yeah, I assumed that earlier as I used to pass in the MongoTemplate to mongock before the 3.2.3 upgrade. I tried use the MongoTemplate based constructor and ran into the index issue you describe. The funny thing is that when I reverted to 2.0.2 I left the mongo template out and it works without passing in the MongoTemplate.

But yes, I do agree that might be the problem for Mongock and am happy to wait for the Index fix.

Cheers

[dieppa]

great, will keep you updated ;)

[Ovaro]

FYI- Jhipster made a decision to stay with mongobee, but it doesn't support Mongo Atlas, hence why I switched (there is an issue on their github issue list if you want to see their reasoning on staying with Mongobee. You might want to convince them to switch to monogock - you would immediately get thousands of users. Just a thought.

[dieppa]

Thanks @Ovaro. Mongobee not only doesn't support Atlas, they don't provide a decent lock mechanism neither a lot of other important features Mongock gives you. But more important than that, they stopped maintaining the library. Actually Mongock started because of a PR that they never reviewed. Since then, we have added a lot of more functionalities and much more we have in our roadmap, which will be published in the next weeks together with our plan to take Mongock to the next level.

But we already know their reasons to not move to Mongock. They are afraidthat we might stop the maintaining Mongock too. However when they decided that, more than a year ago, Mongock had just 11 stars(one of their reasons) now it has 93. I know it's still not much, but I haven't pushed it yet and we not only have continued maintaining the project, we got ambitious plans for it.
But you may like to know that we already had some conversations with them a while ago and agreed to chat again. I will approach them very soon.

Thanks a lot!

[dieppa]

Hello @Ovaro ,

I said that we were able to release a version today, but I need to say sorry because that won't be possible.

I think I will be able to have it done by tomorrow, but then we need to perform some tests, so it will be probably Monday.

Regards

[Ovaro]

@dieppa No worries! I have rolled back to 2.0.2 so this bug hasn't slowed me down. Cheers

[dieppa]

Hey @Ovaro ,

version 3.2.4 is released and should work for you.

Please let me know.
Thanks

[Ovaro]

@dieppa Just tested with the new MongoTemplate constructor and works perfectly.

Thanks!!

[dieppa]

Thank you. I close this :)