ClassCastException

[snuson]

Following the steps in setting up apm and restarting tomcat we get a nasty exception message in our log:

java.lang.ClassCastException: datadog.trace.instrumentation.java.concurrent.ExecutorInstrumentation$CallableWrapper cannot be cast to company.common.asynchronous.immediatetasking.TaskWrapper company.common.asynchronous.immediatetasking.WrapperFutureTask.<init>(WrapperFutureTask.java:42) company.common.asynchronous.immediatetasking.TaskExecutorThreadPool.newTaskFor(TaskExecutorThreadPool.java:57) java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:133) company.common.asynchronous.immediatetasking.TaskExecutorThreadPool.submit(TaskExecutorThreadPool.java:48) company.common.asynchronous.immediatetasking.TaskExecutor.addTask(TaskExecutor.java:246) company.common.asynchronous.immediatetasking.TaskExecutor.addTask(TaskExecutor.java:218) company.common.asynchronous.immediatetasking.TaskExecutor.execute(TaskExecutor.java:99) company.service.cache.IntervalCache.updateCacheIfNecessary(IntervalCache.java:51) company.service.helptexts.HelpTextCache.getHelpLinks(HelpTextCache.java:63) company.service.helptexts.HelpTextServiceImplementation.getHelpLinks(HelpTextServiceImplementation.java:51) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:498) ...

Before over committing to debugging I figured maybe the problem is in datadogs end.

[mar-kolya]

Hi @snusontable! Thanks for your bug report!

It turns out that we may have been working on a problem that is similar to yours. Could you please try this snapshot version of the agent: https://4867-89221572-gh.circle-artifacts.com/0/home/circleci/dd-trace-java/libs/dd-java-agent-0.10.0-SNAPSHOT.jar and let is know if that resolves your problem.

Thanks!

[snuson]

Hey.
Thanks for the incredibly fast response. Unfortunately we still get the exception and an error:

EXCEPTION java.lang.ClassCastException: datadog.trace.instrumentation.java.concurrent.ExecutorInstrumentation$CallableWrapper cannot be cast to company.common.asynchronous.immediatetasking.TaskWrapper company.common.asynchronous.immediatetasking.WrapperFutureTask.<init>(WrapperFutureTask.java:42) company.common.asynchronous.immediatetasking.TaskExecutorThreadPool.newTaskFor(TaskExecutorThreadPool.java:57) java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:133) company.common.asynchronous.immediatetasking.TaskExecutorThreadPool.submit(TaskExecutorThreadPool.java:48) company.common.asynchronous.immediatetasking.TaskExecutor.addTask(TaskExecutor.java:246) company.common.asynchronous.immediatetasking.TaskExecutor.addTask(TaskExecutor.java:218) company.common.asynchronous.immediatetasking.TaskExecutor.execute(TaskExecutor.java:99)...

and

Error logging in. Error: java.lang.ClassCastException: datadog.trace.instrumentation.java.concurrent.ExecutorInstrumentation$CallableWrapper cannot be cast to company.common.asynchronous.immediatetasking.TaskWrapper

[mar-kolya]

Hi. Thanks for your response.

I'm sorry to hear that fix we are currently working on doesn't solve your problem.

So it looks like some of the instrumentation DataDog Java agent does may indeed be in conflict with your code. More specifically DataDog Java agent wraps all Runnables and Callables in ExecutorInstrumentation${Runnable,Callable}Wrapper in order to be able to trace those calls. It looks like some of your code expects your specific instances of TaskWrapper and DataDog's wrapping is causing problems.

It would be helpful to see relevant parts of company.common.asynchronous.immediatetasking.TaskExecutor, company.common.asynchronous.immediatetasking.TaskExecutorThreadPool, company.common.asynchronous.immediatetasking.WrapperFutureTask and company.common.asynchronous.immediatetasking.TaskWrapper - which ever parts you may share to be able to better understand your usage and potentially come up with the solution.

Thanks!

[codylerum]

Running into the same issue now after moving from 0.7.0 -> 0.12.0

This is running in a Wildfly 11 container.

2018-08-09 14:11:56,042 SEVERE [org.glassfish.enterprise.concurrent] (EE-ManagedExecutorService-default-Thread-83) java.lang.ClassCastException: datadog.trace.instrumentation.java.concurrent.ExecutorInstrumentation$RunnableWrapper cannot be cast to org.glassfish.enterprise.concurrent.internal.ManagedFutureTask

I've rolled back to 0.7.0 as this was stopping my managed executor from running

[codylerum]

This is a very standardized Java EE way of executing code.

ManagedExecutorService

Simply calling ManagedExecutorService#execute and passing in a call that implements Runnable.

[tylerbenson]

@codylerum Thanks for the report. We will look into it. It would be helpful if you could provide a snippet of code with how you're interacting with ManagedExecutorService (construct, call into, etc)

[codylerum]

@tylerbenson Super simple

https://gist.github.com/codylerum/5341b1f1703708558be255e67ab18ff0

[asidorenko-emburse]

Running into the same issue here, on Wildfly 10.1 and 0.13.0 of the agent:

java.lang.ClassCastException: datadog.trace.instrumentation.java.concurrent.ExecutorInstrumentation$RunnableWrapper cannot be cast to org.glassfish.enterprise.concurrent.internal.ManagedFutureTask

Our interaction with ManagedExecutorService is also a very simple one:

executor.execute( code );

[realark]

Thanks @codylerum @asidorenko-nexonia. We're working on a long-term fix. It's a top priority, but the change is deep so it may be a few releases before it's ready.

For now the best workaround is to disable the concurrent instrumentation: -Ddd.integration.java_concurrent.enabled=false

[codylerum]

@realark Any ETA on this? At this point we are just hanging back on the 0.7.0 release

[tylerbenson]

@codylerum we made some improvements in 0.16.0 that may improve things for your specific case (3d78afb) but we are actively working on a longterm solution. (See latest work here: #564)

Either way, you can still apply the setting that @realark mentioned to disable the executor instrumentation. That should allow you to update either way.

[realark]

@codylerum @asidorenko-nexonia Thanks for your patience. We have a fix out in our latest 0.19.0 release. I'd appreciate it if you could try running with the fix yourselves. We've done a lot of testing on our end but the instrumentation is pretty generic so there may be an environment we didn't consider.

[tylerbenson]

@codylerum @asidorenko-nexonia any chance you were able to verify the fix in the latest versions? I'd like to close out this issue.