Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MongoConverter support in Spring Modules #87

Closed
savvadem opened this issue Sep 10, 2019 · 21 comments
Closed

MongoConverter support in Spring Modules #87

savvadem opened this issue Sep 10, 2019 · 21 comments

Comments

@savvadem
Copy link

savvadem commented Sep 10, 2019

Hi,
I have some problems with mongock.
It seems that MongoTemplateDecoratorImpl doesn't catch

    @Bean
    public MongoCustomConversions mongoCustomConversions() {
        return new MongoCustomConversions(Arrays.asList(
                new BigDecimalDecimal128Converter(),
                new Decimal128BigDecimalConverter()
        ));

    }

and in the end i receive this exception:

Caused by: com.github.cloudyrock.mongock.MongockException: No converter found capable of converting from type [org.bson.types.Decimal128] to type [java.math.BigDecimal]
	at com.github.cloudyrock.mongock.Mongock.executeMigration(Mongock.java:119) ~[mongock-core-3.0.2.BETA.jar:na]
	at com.github.cloudyrock.mongock.Mongock.execute(Mongock.java:62) ~[mongock-core-3.0.2.BETA.jar:na]
	at com.github.cloudyrock.mongock.SpringBootMongock.run(SpringBootMongock.java:40) ~[mongock-spring-3.0.2.BETA.jar:na]
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:809) [spring-boot-2.0.5.RELEASE.jar:2.0.5.RELEASE]
	... 5 common frames omitted
Caused by: java.lang.reflect.InvocationTargetException: null
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_144]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_144]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_144]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_144]
	at com.github.cloudyrock.mongock.SpringBootMongock.executeChangeSetMethod(SpringBootMongock.java:66) ~[mongock-spring-3.0.2.BETA.jar:na]
	at com.github.cloudyrock.mongock.Mongock.executeIfNewOrRunAlways(Mongock.java:128) ~[mongock-core-3.0.2.BETA.jar:na]
	at com.github.cloudyrock.mongock.Mongock.executeMigration(Mongock.java:112) ~[mongock-core-3.0.2.BETA.jar:na]
	... 8 common frames omitted
Caused by: org.springframework.core.convert.ConverterNotFoundException: No converter found capable of converting from type [org.bson.types.Decimal128] to type [java.math.BigDecimal]
	at org.springframework.core.convert.support.GenericConversionService.handleConverterNotFound(GenericConversionService.java:321) ~[spring-core-5.0.9.RELEASE.jar:5.0.9.RELEASE]
	at org.springframework.core.convert.support.GenericConversionService.convert(GenericConversionService.java:194) ~[spring-core-5.0.9.RELEASE.jar:5.0.9.RELEASE]
	at org.springframework.core.convert.support.GenericConversionService.convert(GenericConversionService.java:174) ~[spring-core-5.0.9.RELEASE.jar:5.0.9.RELEASE]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.getPotentiallyConvertedSimpleRead(MappingMongoConverter.java:908) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.readValue(MappingMongoConverter.java:1412) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter$MongoDbPropertyValueProvider.getPropertyValue(MappingMongoConverter.java:1355) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.readProperties(MappingMongoConverter.java:336) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:292) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:245) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:194) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:190) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:78) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at org.springframework.data.mongodb.core.MongoTemplate$ReadDocumentCallback.doWith(MongoTemplate.java:2800) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at org.springframework.data.mongodb.core.MongoTemplate.executeFindMultiInternal(MongoTemplate.java:2463) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at org.springframework.data.mongodb.core.MongoTemplate.findAll(MongoTemplate.java:1650) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at com.github.cloudyrock.mongock.decorator.impl.MongoTemplateDecoratorImpl.lambda$findAll$61(MongoTemplateDecoratorImpl.java:420) ~[mongock-spring-3.0.2.BETA.jar:na]
	at com.github.cloudyrock.mongock.decorator.util.MethodInvokerImpl.invoke(MethodInvokerImpl.java:18) ~[mongock-core-3.0.2.BETA.jar:na]
	at com.github.cloudyrock.mongock.decorator.impl.MongoTemplateDecoratorImpl.findAll(MongoTemplateDecoratorImpl.java:420) ~[mongock-spring-3.0.2.BETA.jar:na]
	at org.springframework.data.mongodb.core.MongoTemplate.findAll(MongoTemplate.java:1645) ~[spring-data-mongodb-2.0.10.RELEASE.jar:2.0.10.RELEASE]
	at com.github.cloudyrock.mongock.decorator.impl.MongoTemplateDecoratorImpl.lambda$findAll$60(MongoTemplateDecoratorImpl.java:415) ~[mongock-spring-3.0.2.BETA.jar:na]
	at com.github.cloudyrock.mongock.decorator.util.MethodInvokerImpl.invoke(MethodInvokerImpl.java:18) ~[mongock-core-3.0.2.BETA.jar:na]
	at com.github.cloudyrock.mongock.decorator.impl.MongoTemplateDecoratorImpl.findAll(MongoTemplateDecoratorImpl.java:415) ~[mongock-spring-3.0.2.BETA.jar:na]

dependency version:
compile 'com.github.cloudyrock.mongock:mongock-spring:3.0.2.BETA'
Spring Boot 2.0.5.RELEASE

@savvadem
Copy link
Author

As workaround, i used aggregation and bson.Document, so i didn't need converters.

@dieppa
Copy link
Member

dieppa commented Sep 12, 2019

thanks, we'll update you on this.

@dieppa
Copy link
Member

dieppa commented Sep 23, 2019

Hi, we are quite busy right now. is this fix quite important for you or can wait?

@savvadem
Copy link
Author

Sure, i can wait, we are in pre-prod stage right now.

@savvadem
Copy link
Author

Any updates here?

@dieppa
Copy link
Member

dieppa commented Oct 31, 2019

Hi @savvadem , I haven't able to look at this. As you had a workaround we have prioritise other things, like bugs, etc. Version 3 has been a major change internally, that's the reason we're still in BETA.

I cannot tell you this will be fixed this week, but I will try to have a look to at least understand what we are talking about.

Thanks

@dieppa dieppa changed the title MongoTemplateDecoratorImpl inject MongoConverter MongoConverter support in Spring Modules Jan 3, 2020
@Ovaro
Copy link

Ovaro commented Feb 11, 2020

I am having the same issue after upgrading to the non-BETA version 3.2.3 from 2.0.2. I am using Running with Spring Boot v2.1.11.RELEASE, Spring v5.1.12.RELEASE.

Exception:

Caused by: com.github.cloudyrock.mongock.MongockException: Can't find a codec for class java.time.ZonedDateTime.
at com.github.cloudyrock.mongock.Mongock.executeMigration(Mongock.java:118)
at com.github.cloudyrock.mongock.Mongock.execute(Mongock.java:62)
at com.github.cloudyrock.mongock.SpringBootMongock.run(SpringBootMongock.java:35)
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:772)
... 8 common frames omitted
Caused by: java.lang.reflect.InvocationTargetException: null
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.github.cloudyrock.mongock.SpringBootMongock.executeChangeSetMethod(SpringBootMongock.java:60)
at com.github.cloudyrock.mongock.Mongock.executeIfNewOrRunAlways(Mongock.java:127)
at com.github.cloudyrock.mongock.Mongock.executeMigration(Mongock.java:111)
... 11 common frames omitted
Caused by: org.bson.codecs.configuration.CodecConfigurationException: Can't find a codec for class java.time.ZonedDateTime.
at org.bson.codecs.configuration.CodecCache.getOrThrow(CodecCache.java:46)
at org.bson.codecs.configuration.ProvidersCodecRegistry.get(ProvidersCodecRegistry.java:63)
at org.bson.codecs.configuration.ChildCodecRegistry.get(ChildCodecRegistry.java:51)
at org.bson.codecs.DocumentCodec.writeValue(DocumentCodec.java:184)
at org.bson.codecs.DocumentCodec.writeMap(DocumentCodec.java:199)
at org.bson.codecs.DocumentCodec.encode(DocumentCodec.java:141)
at org.bson.codecs.DocumentCodec.encode(DocumentCodec.java:45)
at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:63)
at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:29)
at com.mongodb.operation.BulkWriteBatch$WriteRequestEncoder.encode(BulkWriteBatch.java:387)
at com.mongodb.operation.BulkWriteBatch$WriteRequestEncoder.encode(BulkWriteBatch.java:377)
at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:63)
at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:29)
at com.mongodb.internal.connection.BsonWriterHelper.writeDocument(BsonWriterHelper.java:75)
at com.mongodb.internal.connection.BsonWriterHelper.writePayload(BsonWriterHelper.java:59)
at com.mongodb.internal.connection.CommandMessage.encodeMessageBodyWithMetadata(CommandMessage.java:143)
at com.mongodb.internal.connection.RequestMessage.encode(RequestMessage.java:138)
at com.mongodb.internal.connection.CommandMessage.encode(CommandMessage.java:57)
at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:244)
at com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:99)
at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:444)
at com.mongodb.internal.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:72)
at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:200)
at com.mongodb.internal.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:269)
at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:131)
at com.mongodb.operation.MixedBulkWriteOperation.executeCommand(MixedBulkWriteOperation.java:418)
at com.mongodb.operation.MixedBulkWriteOperation.executeBulkWriteBatch(MixedBulkWriteOperation.java:256)
at com.mongodb.operation.MixedBulkWriteOperation.access$700(MixedBulkWriteOperation.java:67)
at com.mongodb.operation.MixedBulkWriteOperation$1.call(MixedBulkWriteOperation.java:200)
at com.mongodb.operation.MixedBulkWriteOperation$1.call(MixedBulkWriteOperation.java:191)
at com.mongodb.operation.OperationHelper.withReleasableConnection(OperationHelper.java:424)
at com.mongodb.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:191)
at com.mongodb.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:67)
at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:193)
at com.mongodb.client.internal.MongoCollectionImpl.executeSingleWriteRequest(MongoCollectionImpl.java:960)
at com.mongodb.client.internal.MongoCollectionImpl.executeInsertOne(MongoCollectionImpl.java:494)
at com.mongodb.client.internal.MongoCollectionImpl.insertOne(MongoCollectionImpl.java:478)
at com.mongodb.client.internal.MongoCollectionImpl.insertOne(MongoCollectionImpl.java:472)
at com.github.cloudyrock.mongock.decorator.MongoCollectionDecorator.lambda$insertOne$15(MongoCollectionDecorator.java:158)
at com.github.cloudyrock.mongock.decorator.util.MethodInvokerImpl.invoke(MethodInvokerImpl.java:24)
at com.github.cloudyrock.mongock.decorator.MongoCollectionDecorator.insertOne(MongoCollectionDecorator.java:158)
at com.github.cloudyrock.mongock.decorator.MongoCollectionDecorator.lambda$insertOne$15(MongoCollectionDecorator.java:158)
at com.github.cloudyrock.mongock.decorator.util.MethodInvokerImpl.invoke(MethodInvokerImpl.java:24)
at com.github.cloudyrock.mongock.decorator.MongoCollectionDecorator.insertOne(MongoCollectionDecorator.java:158)
at com.github.cloudyrock.mongock.decorator.MongoCollectionDecorator.lambda$insertOne$15(MongoCollectionDecorator.java:158)
at com.github.cloudyrock.mongock.decorator.util.MethodInvokerImpl.invoke(MethodInvokerImpl.java:24)
at com.github.cloudyrock.mongock.decorator.MongoCollectionDecorator.insertOne(MongoCollectionDecorator.java:158)
at org.springframework.data.mongodb.core.MongoTemplate$7.doInCollection(MongoTemplate.java:1498)
at org.springframework.data.mongodb.core.MongoTemplate.execute(MongoTemplate.java:544)
at com.github.cloudyrock.mongock.decorator.impl.MongoTemplateDecoratorImpl.lambda$execute$8(MongoTemplateDecoratorImpl.java:145)
at com.github.cloudyrock.mongock.decorator.util.MethodInvokerImpl.invoke(MethodInvokerImpl.java:18)
at com.github.cloudyrock.mongock.decorator.impl.MongoTemplateDecoratorImpl.execute(MongoTemplateDecoratorImpl.java:145)
at org.springframework.data.mongodb.core.MongoTemplate.saveDocument(MongoTemplate.java:1488)
at org.springframework.data.mongodb.core.MongoTemplate.doSave(MongoTemplate.java:1423)
at org.springframework.data.mongodb.core.MongoTemplate.save(MongoTemplate.java:1373)
at org.springframework.data.mongodb.core.MongoTemplate.save(MongoTemplate.java:1359)
at io.ovaro.config.dbmigrations.InitialSetupMigration.job1UnassignedFixedManual(InitialSetupMigration.java:2646)
at io.ovaro.config.dbmigrations.InitialSetupMigration.addJobs(InitialSetupMigration.java:2338)
at io.ovaro.config.dbmigrations.InitialSetupMigration.addJobs(InitialSetupMigration.java:2328)
... 18 common frames omitted

@dieppa
Copy link
Member

dieppa commented Feb 11, 2020

Sorry , I didn't update on this. I wasn't able to reproduce this, and we was distracted by some other more urgent matters. We'll look at this today. However, it would be nice if you can provide an example app that I can checkout out and reproduce easily.

Thanks @Ovaro

@Ovaro
Copy link

Ovaro commented Feb 11, 2020

Hi @dieppa - this is on a Jhipster based app where I have moved to monogock rather than mongobee.

@Configuration @EnableMongoRepositories(basePackages = "xxx.repository", includeFilters = @Filter(type = FilterType.ASSIGNABLE_TYPE, value = MongoRepository.class)) @Profile("!" + JHipsterConstants.SPRING_PROFILE_CLOUD) @Import(value = MongoAutoConfiguration.class) @EnableMongoAuditing(auditorAwareRef = "springSecurityAuditorAware") public class DatabaseConfiguration {

private final Logger log = LoggerFactory.getLogger(DatabaseConfiguration.class);

@Bean
public ValidatingMongoEventListener validatingMongoEventListener() {
    return new ValidatingMongoEventListener(validator());
}

@Bean
public LocalValidatorFactoryBean validator() {
    return new LocalValidatorFactoryBean();
}

@Bean
public MongoCustomConversions customConversions() {
    log.debug("Adding custom conversions ZZZ");
    List<Converter<?, ?>> converters = new ArrayList<>();
    converters.add(DateToZonedDateTimeConverter.INSTANCE);
    converters.add(ZonedDateTimeToDateConverter.INSTANCE);
    return new MongoCustomConversions(converters);
}

@Bean
public SpringBootMongock mongock(ApplicationContext springContext, MongoClient mongoClient, MongoTemplate mongoTemplate, MongoProperties mongoProperties) {
    return new SpringBootMongockBuilder(mongoClient, mongoProperties.getMongoClientDatabase(), "xxxx.config.dbmigrations")
        .setApplicationContext(springContext)
        .setLockQuickConfig()
        .build();
}

// The following converter definitions to fix org.bson.codecs.configuration.CodecConfigurationException: Can't find a codec for class java.time.ZonedDateTime

@WritingConverter
public enum ZonedDateTimeToDateConverter implements Converter<ZonedDateTime, Date> {
    INSTANCE;

    @Override
    public Date convert(ZonedDateTime source) {
        return source == null ? null : Date.from(source.toInstant());
    }
}

@ReadingConverter
public enum DateToZonedDateTimeConverter implements Converter<Date, ZonedDateTime> {
    INSTANCE;

    @Override
    public ZonedDateTime convert(Date source) {
        return source == null ? null : ZonedDateTime.ofInstant(source.toInstant(), ZoneId.systemDefault());
    }
}

}

And the example of a class that doesn't work:

@document(collection = "job")
public class Job {
@id
private String id;
private String title;
@NotNull
private String state;

@NotNull
private ZonedDateTime lastUpdated;
@NotNull
private ZonedDateTime posted;

....
}

This all worked in version 2.0.2 but has stopped in version 3 and above. Mongock is being created in exactly the same way so have the converters registered (those debug log lines stating the converter is registered occurs way before the errors).

@dieppa
Copy link
Member

dieppa commented Feb 11, 2020

Hello, that's probably because we migrate from a proxy approach to control the lock to implement decorators.

Anyway, we have been able to reproduce it and doesn't seem that will be hard to fix. It will be fixed by Thursday evening.

Cheers

@Ovaro
Copy link

Ovaro commented Feb 11, 2020

@dieppa Awesome! Great to hear it wasn't a big problem. I didn't look into the cause as I saw this open issue. Really appreciated. Cheers.

@dieppa
Copy link
Member

dieppa commented Feb 11, 2020

@Ovaro, I believe that if you use MongoTemplate, instead of MongoClient, it works fine. Can you check, please?

Just notice that currently there is a bug using MongoTemplate, which affects for example to fields with annotated with @indexed. However this is fixed, but not released yet. Will be released before Thursday, together with any fix this issue require, if it's the case.

@Ovaro
Copy link

Ovaro commented Feb 11, 2020

@dieppa Yeah, I assumed that earlier as I used to pass in the MongoTemplate to mongock before the 3.2.3 upgrade. I tried use the MongoTemplate based constructor and ran into the index issue you describe. The funny thing is that when I reverted to 2.0.2 I left the mongo template out and it works without passing in the MongoTemplate.

But yes, I do agree that might be the problem for Mongock and am happy to wait for the Index fix.

Cheers

@dieppa
Copy link
Member

dieppa commented Feb 11, 2020

great, will keep you updated ;)

@Ovaro
Copy link

Ovaro commented Feb 11, 2020

FYI- Jhipster made a decision to stay with mongobee, but it doesn't support Mongo Atlas, hence why I switched (there is an issue on their github issue list if you want to see their reasoning on staying with Mongobee. You might want to convince them to switch to monogock - you would immediately get thousands of users. Just a thought.

@dieppa
Copy link
Member

dieppa commented Feb 11, 2020

Thanks @Ovaro. Mongobee not only doesn't support Atlas, they don't provide a decent lock mechanism neither a lot of other important features Mongock gives you. But more important than that, they stopped maintaining the library. Actually Mongock started because of a PR that they never reviewed. Since then, we have added a lot of more functionalities and much more we have in our roadmap, which will be published in the next weeks together with our plan to take Mongock to the next level.

But we already know their reasons to not move to Mongock. They are afraidthat we might stop the maintaining Mongock too. However when they decided that, more than a year ago, Mongock had just 11 stars(one of their reasons) now it has 93. I know it's still not much, but I haven't pushed it yet and we not only have continued maintaining the project, we got ambitious plans for it.
But you may like to know that we already had some conversations with them a while ago and agreed to chat again. I will approach them very soon.

Thanks a lot!

@dieppa
Copy link
Member

dieppa commented Feb 13, 2020

Hello @Ovaro ,

I said that we were able to release a version today, but I need to say sorry because that won't be possible.

I think I will be able to have it done by tomorrow, but then we need to perform some tests, so it will be probably Monday.

Regards

@Ovaro
Copy link

Ovaro commented Feb 13, 2020

@dieppa No worries! I have rolled back to 2.0.2 so this bug hasn't slowed me down. Cheers

@dieppa
Copy link
Member

dieppa commented Feb 17, 2020

Hey @Ovaro ,

version 3.2.4 is released and should work for you.

Please let me know.
Thanks

@Ovaro
Copy link

Ovaro commented Feb 17, 2020

@dieppa Just tested with the new MongoTemplate constructor and works perfectly.

Thanks!!

@dieppa
Copy link
Member

dieppa commented Feb 17, 2020

Thank you. I close this :)

@dieppa dieppa closed this as completed Feb 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants