Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cassandraql: itests failing on quarkus-platform #3621

Closed
aldettinger opened this issue Mar 15, 2022 · 12 comments
Closed

cassandraql: itests failing on quarkus-platform #3621

aldettinger opened this issue Mar 15, 2022 · 12 comments
Assignees
Milestone

Comments

@aldettinger
Copy link
Contributor

While attempting to upgrade quarkus-platform to camel-quarkus-2.8.0-M1, the cassandraql tests are failing:


2022-03-15 09:10:25,286 ERROR [org.apa.cam.imp.eng.AbstractCamelContext] (main) Error starting CamelContext (camel-1) due to exception thrown: Failed to start route route5 because of null: org.apache.camel.FailedToStartRouteException: Failed to start route route5 because of null
	at org.apache.camel.impl.engine.RouteService.warmUp(RouteService.java:123)
	at org.apache.camel.impl.engine.InternalRouteStartupManager.doWarmUpRoutes(InternalRouteStartupManager.java:306)
	at org.apache.camel.impl.engine.InternalRouteStartupManager.safelyStartRouteServices(InternalRouteStartupManager.java:189)
	at org.apache.camel.impl.engine.InternalRouteStartupManager.doStartOrResumeRoutes(InternalRouteStartupManager.java:147)
	at org.apache.camel.impl.engine.AbstractCamelContext.doStartCamel(AbstractCamelContext.java:3309)
	at org.apache.camel.impl.engine.AbstractCamelContext.doStartContext(AbstractCamelContext.java:2988)
	at org.apache.camel.impl.engine.AbstractCamelContext.doStart(AbstractCamelContext.java:2942)
	at org.apache.camel.support.service.BaseService.start(BaseService.java:119)
	at org.apache.camel.impl.engine.AbstractCamelContext.start(AbstractCamelContext.java:2604)
	at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:247)
	at org.apache.camel.quarkus.main.CamelMain.doStart(CamelMain.java:94)
	at org.apache.camel.support.service.BaseService.start(BaseService.java:119)
	at org.apache.camel.quarkus.main.CamelMain.startEngine(CamelMain.java:140)
	at org.apache.camel.quarkus.main.CamelMainRuntime.start(CamelMainRuntime.java:49)
	at org.apache.camel.quarkus.core.CamelBootstrapRecorder.start(CamelBootstrapRecorder.java:45)
	at io.quarkus.deployment.steps.CamelBootstrapProcessor$boot173480958.deploy_0(Unknown Source)
	at io.quarkus.deployment.steps.CamelBootstrapProcessor$boot173480958.deploy(Unknown Source)
	at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
	at io.quarkus.runtime.Application.start(Application.java:101)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at io.quarkus.runner.bootstrap.StartupActionImpl.run(StartupActionImpl.java:237)
	at io.quarkus.test.junit.QuarkusTestExtension.doJavaStart(QuarkusTestExtension.java:249)
	at io.quarkus.test.junit.QuarkusTestExtension.ensureStarted(QuarkusTestExtension.java:608)
	at io.quarkus.test.junit.QuarkusTestExtension.beforeAll(QuarkusTestExtension.java:646)
	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$invokeBeforeAllCallbacks$10(ClassBasedTestDescriptor.java:381)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.invokeBeforeAllCallbacks(ClassBasedTestDescriptor.java:381)
	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.before(ClassBasedTestDescriptor.java:205)
	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.before(ClassBasedTestDescriptor.java:80)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:148)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
	at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
	at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:142)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:117)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:383)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:344)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:417)
Caused by: org.apache.camel.RuntimeCamelException: java.lang.IllegalArgumentException: Can't find class org.apache.camel.quarkus.component.cassandraql.it.CustomLoadBalancingPolicy (specified by basic.load-balancing-policy.class)
	at org.apache.camel.RuntimeCamelException.wrapRuntimeCamelException(RuntimeCamelException.java:51)
	at org.apache.camel.support.ChildServiceSupport.start(ChildServiceSupport.java:67)
	at org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:113)
	at org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:130)
	at org.apache.camel.impl.engine.DefaultChannel.doStart(DefaultChannel.java:126)
	at org.apache.camel.support.service.BaseService.start(BaseService.java:119)
	at org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:113)
	at org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:116)
	at org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:130)
	at org.apache.camel.processor.Pipeline.doStart(Pipeline.java:221)
	at org.apache.camel.support.service.BaseService.start(BaseService.java:119)
	at org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:113)
	at org.apache.camel.support.processor.DelegateAsyncProcessor.doStart(DelegateAsyncProcessor.java:89)
	at org.apache.camel.support.service.BaseService.start(BaseService.java:119)
	at org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:113)
	at org.apache.camel.impl.engine.RouteService.startChildServices(RouteService.java:396)
	at org.apache.camel.impl.engine.RouteService.doWarmUp(RouteService.java:193)
	at org.apache.camel.impl.engine.RouteService.warmUp(RouteService.java:121)
	... 67 more
Caused by: java.lang.IllegalArgumentException: Can't find class org.apache.camel.quarkus.component.cassandraql.it.CustomLoadBalancingPolicy (specified by basic.load-balancing-policy.class)
	at com.datastax.oss.driver.internal.core.util.Reflection.resolveClass(Reflection.java:302)
	at com.datastax.oss.driver.internal.core.util.Reflection.buildFromConfig(Reflection.java:235)
	at com.datastax.oss.driver.internal.core.util.Reflection.buildFromConfigProfiles(Reflection.java:194)
	at com.datastax.oss.driver.internal.core.context.DefaultDriverContext.buildLoadBalancingPolicies(DefaultDriverContext.java:349)
	at com.datastax.oss.driver.internal.core.util.concurrent.LazyReference.get(LazyReference.java:55)
	at com.datastax.oss.driver.internal.core.context.DefaultDriverContext.getLoadBalancingPolicies(DefaultDriverContext.java:755)
	at com.datastax.oss.driver.internal.core.session.DefaultSession$SingleThreaded.init(DefaultSession.java:338)
	at com.datastax.oss.driver.internal.core.session.DefaultSession$SingleThreaded.access$1100(DefaultSession.java:300)
	at com.datastax.oss.driver.internal.core.session.DefaultSession.lambda$init$0(DefaultSession.java:146)
	at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)
	at io.netty.util.concurrent.PromiseTask.run(PromiseTask.java:106)
	at io.netty.channel.DefaultEventLoop.run(DefaultEventLoop.java:54)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:829)

Those itests are failing only on quarkus-platform, it works on cq main and even cq-2.8.x + quarkus 2.7.4.Final.

This class org.apache.camel.quarkus.component.cassandraql.it.CustomLoadBalancingPolicy is present in the 2.8.0-M1 itest-jar.

I've experimented with <quarkus.arc.unremovable-types>org.apache.camel.quarkus.component.cassandraql.it.CustomLoadBalancingPolicy</quarkus.arc.unremovable-types> to no avail.

While checking the dependency tree, I've noticed that quarkus-platform forces quarkus version to 2.7.4.Final while 2.8.0-M1 is built on 2.7.3.Final. I don't think it's an issue but worth mentioning.

I will skip the test on quarkus-platform as I have no more ideas.
@jamesnetherton in case it rings any bell

@jamesnetherton
Copy link
Contributor

It could be because of this:

It's probably safe to remove references to that annotation in CustomLoadBalancingPolicy. One of those cases where I lazily let my IDE produce the method overrides...

aldettinger added a commit to aldettinger/camel-quarkus that referenced this issue Mar 15, 2022
@aldettinger aldettinger self-assigned this Mar 15, 2022
@aldettinger
Copy link
Contributor Author

I've raised a PR #3623.

I would be interested in any pointers explaining why @ NonNull would make the class unloadable in quarkus-platform.
Maybe there is some kind of reflective class replacement under the hood. And that would work in main and not quarkus-platform.

@aldettinger
Copy link
Contributor Author

Backported to 2.7.x: 96bd3a6

@aldettinger aldettinger added this to the 2.8.0 milestone Mar 15, 2022
@jamesnetherton
Copy link
Contributor

Seems this did not fix the problem. I see the same thing on the 2.7.1 and 2.8.0 platform upgrades.

@jamesnetherton jamesnetherton reopened this Apr 7, 2022
@aldettinger
Copy link
Contributor Author

@jamesnetherton I don't have any idea what it could be. Would you like to have a try investigating this ticket ?

@jamesnetherton
Copy link
Contributor

I will try but I have no idea what's going on either 🙂

@jamesnetherton jamesnetherton modified the milestones: 2.8.0, 2.9.0 Apr 7, 2022
@aldettinger
Copy link
Contributor Author

There is a thing that also changed in 2.7.1 and 2.8.0. It is that camel-quarkus-bom does not import quarkus-bom anymore.

I don't see how it relate and I hope it's not a confusing information. It's just the only thing that pop to my mind 🤔

@jamesnetherton
Copy link
Contributor

jamesnetherton commented Apr 7, 2022

I guess the thing to try is whether it fails with a generated project from code.quarkus.io. It it ends up working ok, then we know it's something odd about the test setup on the platform.

@zbendhiba
Copy link
Contributor

It's cool by the way to open these kinds of issues, as we sometimes tend to disable some tests and forget to investigate

@jamesnetherton
Copy link
Contributor

Turns out this is a class loading issue that only occurs with the itests. I was forced to set quarkus.test.flat-class-path=true in the tests to get object deserialization to work properly. It seems this is not honored (or does not work) when run on the platform.

This problem only occurs at test time. When running the JAR or native application, it works fine.

We can make some amendments to the Cassandra Camel component to avoid setting quarkus.test.flat-class-path.

  1. Explicitly provide a ClassLoader to CqlSessionBuilder
  2. Modify CassandraCamelCodec to use ClassLoadingAwareObjectInputStream

@aldettinger
Copy link
Contributor Author

Well done. Congrats for this investigation 👍 Moreover, identifying that there is no consequences for users is really cool.

@jamesnetherton jamesnetherton modified the milestones: 2.9.0, 2.10.0 May 5, 2022
jamesnetherton added a commit to jamesnetherton/camel-quarkus that referenced this issue Jun 8, 2022
jamesnetherton added a commit to jamesnetherton/camel-quarkus that referenced this issue Jun 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants