Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Storage provider support for latest elasticsearch #291

Closed
SympathyForTheDev opened this issue Dec 22, 2021 · 5 comments
Closed

[Feature] Storage provider support for latest elasticsearch #291

SympathyForTheDev opened this issue Dec 22, 2021 · 5 comments
Labels
caused by dependency Issue is caused by a dependency enhancement New feature or request
Milestone
5.0.0

Comments

@SympathyForTheDev
Copy link

Are you running the latest version of JobRunr?
Yes

Describe the bug

when i'm using latest version of elasticsearch (7.16.2) with jobrunr 4.0.5

    implementation "org.jobrunr:jobrunr-spring-boot-starter:4.0.5"
    implementation 'org.elasticsearch:elasticsearch:7.16.2'
    implementation 'org.elasticsearch.client:elasticsearch-rest-high-level-client:7.16.2'

I'm getting the following errors when my application startup

2021-12-22 11:39:12.448 ERROR 749039 --- [           main] o.s.boot.SpringApplication               : Application run failed

org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'recurringJobPostProcessor' defined in class path resource [org/jobrunr/spring/autoconfigure/JobRunrAutoConfiguration.class]: Unsatisfied dependency expressed through method 'recurringJobPostProcessor' parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'jobScheduler' defined in class path resource [org/jobrunr/spring/autoconfigure/JobRunrAutoConfiguration.class]: Unsatisfied dependency expressed through method 'jobScheduler' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'storageProvider' defined in class path resource [org/jobrunr/spring/autoconfigure/storage/JobRunrElasticSearchStorageAutoConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.jobrunr.storage.StorageProvider]: Factory method 'elasticSearchStorageProvider' threw exception; nested exception is java.lang.NoClassDefFoundError: org/elasticsearch/common/xcontent/json/JsonXContent
	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:800) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:541) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1352) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1195) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:213) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.context.support.PostProcessorRegistrationDelegate.registerBeanPostProcessors(PostProcessorRegistrationDelegate.java:270) ~[spring-context-5.3.13.jar:5.3.13]
	at org.springframework.context.support.AbstractApplicationContext.registerBeanPostProcessors(AbstractApplicationContext.java:762) ~[spring-context-5.3.13.jar:5.3.13]
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:567) ~[spring-context-5.3.13.jar:5.3.13]
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:145) ~[spring-boot-2.6.1.jar:2.6.1]
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:730) ~[spring-boot-2.6.1.jar:2.6.1]
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:412) ~[spring-boot-2.6.1.jar:2.6.1]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:302) ~[spring-boot-2.6.1.jar:2.6.1]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1301) ~[spring-boot-2.6.1.jar:2.6.1]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1290) ~[spring-boot-2.6.1.jar:2.6.1]
	at com.straumann.flowable.work.StraumannFlowableWork.main(StraumannFlowableWork.java:21) ~[main/:na]
Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'jobScheduler' defined in class path resource [org/jobrunr/spring/autoconfigure/JobRunrAutoConfiguration.class]: Unsatisfied dependency expressed through method 'jobScheduler' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'storageProvider' defined in class path resource [org/jobrunr/spring/autoconfigure/storage/JobRunrElasticSearchStorageAutoConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.jobrunr.storage.StorageProvider]: Factory method 'elasticSearchStorageProvider' threw exception; nested exception is java.lang.NoClassDefFoundError: org/elasticsearch/common/xcontent/json/JsonXContent
	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:800) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:541) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1352) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1195) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1380) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1300) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791) ~[spring-beans-5.3.13.jar:5.3.13]
	... 19 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'storageProvider' defined in class path resource [org/jobrunr/spring/autoconfigure/storage/JobRunrElasticSearchStorageAutoConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.jobrunr.storage.StorageProvider]: Factory method 'elasticSearchStorageProvider' threw exception; nested exception is java.lang.NoClassDefFoundError: org/elasticsearch/common/xcontent/json/JsonXContent
	at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:658) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:638) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1352) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1195) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1380) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1300) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791) ~[spring-beans-5.3.13.jar:5.3.13]
	... 33 common frames omitted
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.jobrunr.storage.StorageProvider]: Factory method 'elasticSearchStorageProvider' threw exception; nested exception is java.lang.NoClassDefFoundError: org/elasticsearch/common/xcontent/json/JsonXContent
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185) ~[spring-beans-5.3.13.jar:5.3.13]
	at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:653) ~[spring-beans-5.3.13.jar:5.3.13]
	... 47 common frames omitted
Caused by: java.lang.NoClassDefFoundError: org/elasticsearch/common/xcontent/json/JsonXContent
	at org.jobrunr.storage.nosql.elasticsearch.ElasticSearchDBCreator.markMigrationAsDone(ElasticSearchDBCreator.java:103) ~[jobrunr-4.0.5.jar:na]
	at org.jobrunr.storage.nosql.common.NoSqlDatabaseCreator.runMigrationIfNecessary(NoSqlDatabaseCreator.java:51) ~[jobrunr-4.0.5.jar:na]
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[na:na]
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1541) ~[na:na]
	at java.base/java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395) ~[na:na]
	at java.base/java.util.stream.Sink$ChainedReference.end(Sink.java:258) ~[na:na]
	at java.base/java.util.stream.Sink$ChainedReference.end(Sink.java:258) ~[na:na]
	at java.base/java.util.stream.Sink$ChainedReference.end(Sink.java:258) ~[na:na]
	at java.base/java.util.stream.Sink$ChainedReference.end(Sink.java:258) ~[na:na]
	at java.base/java.util.stream.Sink$ChainedReference.end(Sink.java:258) ~[na:na]
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485) ~[na:na]
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[na:na]
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[na:na]
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[na:na]
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[na:na]
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) ~[na:na]
	at org.jobrunr.storage.nosql.common.NoSqlDatabaseCreator.runMigrations(NoSqlDatabaseCreator.java:31) ~[jobrunr-4.0.5.jar:na]
	at org.jobrunr.storage.nosql.elasticsearch.ElasticSearchDBCreator.runMigrations(ElasticSearchDBCreator.java:60) ~[jobrunr-4.0.5.jar:na]
	at org.jobrunr.storage.nosql.elasticsearch.ElasticSearchStorageProvider.<init>(ElasticSearchStorageProvider.java:121) ~[jobrunr-4.0.5.jar:na]
	at org.jobrunr.storage.nosql.elasticsearch.ElasticSearchStorageProvider.<init>(ElasticSearchStorageProvider.java:105) ~[jobrunr-4.0.5.jar:na]
	at org.jobrunr.spring.autoconfigure.storage.JobRunrElasticSearchStorageAutoConfiguration.elasticSearchStorageProvider(JobRunrElasticSearchStorageAutoConfiguration.java:26) ~[jobrunr-spring-boot-starter-4.0.5.jar:na]
	at org.jobrunr.spring.autoconfigure.storage.JobRunrElasticSearchStorageAutoConfiguration$$EnhancerBySpringCGLIB$$c76f7f9b.CGLIB$elasticSearchStorageProvider$0(<generated>) ~[jobrunr-spring-boot-starter-4.0.5.jar:na]
	at org.jobrunr.spring.autoconfigure.storage.JobRunrElasticSearchStorageAutoConfiguration$$EnhancerBySpringCGLIB$$c76f7f9b$$FastClassBySpringCGLIB$$17808e39.invoke(<generated>) ~[jobrunr-spring-boot-starter-4.0.5.jar:na]
	at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244) ~[spring-core-5.3.13.jar:5.3.13]
	at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331) ~[spring-context-5.3.13.jar:5.3.13]
	at org.jobrunr.spring.autoconfigure.storage.JobRunrElasticSearchStorageAutoConfiguration$$EnhancerBySpringCGLIB$$c76f7f9b.elasticSearchStorageProvider(<generated>) ~[jobrunr-spring-boot-starter-4.0.5.jar:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.3.13.jar:5.3.13]
	... 48 common frames omitted
Caused by: java.lang.ClassNotFoundException: org.elasticsearch.common.xcontent.json.JsonXContent
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581) ~[na:na]
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) ~[na:na]
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ~[na:na]
	... 79 common frames omitted

it's look like the method

    @Override
    protected boolean markMigrationAsDone(NoSqlMigration noSqlMigration) {
        try {
            XContentBuilder builder = JsonXContent.contentBuilder().prettyPrint();
            builder.startObject();
            builder.field(Migrations.FIELD_NAME, noSqlMigration.getClassName());
            builder.field(Migrations.FIELD_DATE, Instant.now());
            builder.endObject();
            IndexRequest indexRequest = new IndexRequest(migrationIndexName)
                    .id(substringBefore(noSqlMigration.getClassName(), "_"))
                    .setRefreshPolicy(RefreshPolicy.IMMEDIATE)
                    .source(builder);
            return client.index(indexRequest, RequestOptions.DEFAULT) != null;
        } catch (IOException e) {
            throw new StorageException(e);
        }
    }

in ElasticSearchDBCreator, is using removed stuff in the latest version of elasticsearch

Environment
I'm using JobRunr version: 4.0.5
I'm running on JRE / JDK (e.g. OpenJDK 1.8.0_292): JDK11
I'm using the following StorageProvider (e.g. Oracle / MongoDB / ...): elasticsearch 7.16.2

How to Reproduce

use latest elasticsearch version

Expected behavior
Application should startup properly

Additional context
N/A
@SympathyForTheDev SympathyForTheDev changed the title [BUG] Storage provider broken with lastest elasticsearch [BUG] Storage provider autoconfiguration broken with lastest elasticsearch Dec 22, 2021
@rdehuyss
Copy link
Contributor

Happy to accept a PR for this

@rdehuyss
Copy link
Contributor

And I think you should create a bug report also at Elastic - they broke backwards compatibility with a minor version update..

@rdehuyss rdehuyss added the caused by dependency Issue is caused by a dependency label Dec 22, 2021
@rdehuyss rdehuyss changed the title [BUG] Storage provider autoconfiguration broken with lastest elasticsearch [Improvement] Storage provider support for latest elasticsearch Dec 22, 2021
@rdehuyss rdehuyss changed the title [Improvement] Storage provider support for latest elasticsearch [Feature] Storage provider support for latest elasticsearch Dec 22, 2021
@SympathyForTheDev
Copy link
Author

after some testing, jobrunr work well with Elasticsearch up to version 7.15.2

it's look like 7.16.0 introduce the breaking changes
elastic/elasticsearch#78831

@SympathyForTheDev
Copy link
Author

SympathyForTheDev commented Dec 22, 2021 •
edited
Loading

I can do the PR but
It's just a package to change in the JobRunr code but it will break compatibility with previous version of Elasticsearch, I don't know how I can help to handle this

@rdehuyss rdehuyss added the enhancement New feature or request label Dec 24, 2021
@theexiile1305 theexiile1305 mentioned this issue Dec 24, 2021
Closed
@SympathyForTheDev SympathyForTheDev mentioned this issue Jan 3, 2022
Closed
58 tasks
@rdehuyss rdehuyss mentioned this issue Jan 10, 2022
Closed
rdehuyss added a commit that referenced this issue Jan 15, 2022
@rdehuyss
Upgrade dependencies and fix #291
f751de7
@rdehuyss rdehuyss added this to the 5.0.0 milestone Jan 26, 2022
@rdehuyss
Copy link
Contributor

Solved in v5.0.0 branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
caused by dependency Issue is caused by a dependency enhancement New feature or request
Projects
None yet
Milestone
5.0.0
Development

Successfully merging a pull request may close this issue.

[Feature] Support latest version of H2 Database
2 participants
@rdehuyss @SympathyForTheDev