Runtime ClassNotFoundError #35

Closed
FroMage opened this Issue Mar 18, 2013 · 29 comments

Comments

Projects
None yet
5 participants
Owner

FroMage commented Mar 18, 2013

As shown in ceylon/ceylon-compiler#1100 we have a runtime error with the following code:

module foo '1.0.0' {
  import java.base '7';
  import 'commons-codec.commons-codec' '1.4';
  import 'commons-httpclient.commons-httpclient' '3.1';
  import 'org.apache.camel.camel-core' '2.9.4';
  import 'org.apache.camel.camel-jetty' '2.9.4';
  import 'org.apache.camel.camel-http' '2.9.4';
  import 'org.eclipse.jetty.jetty-server' '7.5.4.v20111024';
  import 'org.eclipse.jetty.jetty-client' '7.5.4.v20111024';
  import 'org.eclipse.jetty.jetty-http' '7.5.4.v20111024';
  import 'org.eclipse.jetty.jetty-util' '7.5.4.v20111024';
  import 'org.eclipse.jetty.jetty-io' '7.5.4.v20111024';
  import 'org.eclipse.jetty.jetty-jmx' '7.5.4.v20111024';
  import 'javax.servlet.servlet-api' '2.5';
}
import org.apache.camel.impl { DefaultCamelContext }
import org.apache.camel.component.jetty { JettyHttpComponent }
import org.apache.camel.builder { RouteBuilder }
import java.lang { Thread { currentThread } }

doc "Run the module `simple`."
shared void run() {
  print("Start Camel");
  value context = DefaultCamelContext();
  context.addComponent("jetty", JettyHttpComponent());
  object routeBuilder extends RouteBuilder() {
    shared actual void configure() {
      from("jetty:http://localhost:8080").log("got request");
    }
  }
  context.addRoutes(routeBuilder);
  context.start();
  currentThread().sleep(10000);
  print("Stop Camel");
}

When I connect to localhost:8080 (does not fail before I connect), I get:

Start Camel
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Exception in thread "main" java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at com.redhat.ceylon.launcher.Launcher.run(Launcher.java:68)
    at com.redhat.ceylon.launcher.Launcher.main(Launcher.java:19)
Caused by: java.lang.NoClassDefFoundError: org/eclipse/jetty/server/Connector
    at foo.run_.run(run.ceylon:10)
    at foo.run_.main(run.ceylon)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at ceylon.modules.api.runtime.SecurityActions.invokeRunInternal(SecurityActions.java:58)
    at ceylon.modules.api.runtime.SecurityActions.invokeRun(SecurityActions.java:48)
    at ceylon.modules.api.runtime.AbstractRuntime.invokeRun(AbstractRuntime.java:84)
    at ceylon.modules.api.runtime.AbstractRuntime.execute(AbstractRuntime.java:131)
    at ceylon.modules.api.runtime.AbstractRuntime.execute(AbstractRuntime.java:119)
    at ceylon.modules.Main.execute(Main.java:69)
    at ceylon.modules.Main.main(Main.java:42)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.jboss.modules.Module.run(Module.java:270)
    at org.jboss.modules.Main.main(Main.java:294)
    at ceylon.modules.bootstrap.CeylonRunTool.run(CeylonRunTool.java:151)
    at com.redhat.ceylon.tools.CeylonTool.run(CeylonTool.java:236)
    at com.redhat.ceylon.tools.CeylonTool.bootstrap(CeylonTool.java:205)
    at com.redhat.ceylon.tools.CeylonTool.start(CeylonTool.java:178)
    ... 6 more
Caused by: java.lang.ClassNotFoundException: org.eclipse.jetty.server.Connector from [Module "org.apache.camel.camel-jetty:2.9.4" from Ceylon ModuleLoader: RootRepositoryManager: FileContentStore: /home/stephane/.ceylon/cache]
    at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:190)
    at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:468)
    at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:456)
    at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:398)
    at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:120)
    ... 29 more

Not sure at all why that is: the org.eclipse.jetty.server.Connector class appears to come from the jetty-server module, which appears to be imported by the camel-jetty artifact.

@FroMage FroMage referenced this issue in ceylon/ceylon-compiler Mar 18, 2013

Closed

Failed to import valid java dependency #1100

Member

alesj commented Mar 18, 2013

My guess - as we've seen this before, some other dependency is missing in "jetty-server",
and this one just masks the real cause.

Owner

FroMage commented Mar 18, 2013

Perhaps, but in that case, we need to find a way to unmask these errors, this is important.

Member

davidfestal commented Mar 19, 2013

Even when trying to manually and explicitely add in the module descriptor all the transitive depdendencies I can find, the run finished with this error :

... Caused by: java.lang.ClassNotFoundException: org.eclipse.jetty.util.thread.ThreadPool from [Module "org.apache.camel:camel-jetty:2.10.1" from Ceylon ModuleLoader: RootRepositoryManager: FileContentStore: /home/david/.ceylon/cache] at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:190) at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:468) at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:456) at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:398) at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:120)
it seems it tries to find the ThreadPool class in the camel-jetty module (where it is probably used) and in fact it is defined in org.mortbay.jetty:jetty-utils, that I also added in the deps as shown here :

Debug: -> Found at /org/mortbay/jetty/jetty-util/6.1.26/jetty-util-6.1.26.jar

Other information : the same code is running inside Eclipse using the flat classpath containing the same jars (except when we have 2 versions of the same module)

Owner

FroMage commented Apr 3, 2013

So I found the cause of JBoss modules hiding the exception for us. It tries to load org.eclipse.jetty.server.Connector from jetty-server, which finds it but fails to link and we get to ModulesClassLoader.loadClassLocal(...) which gets a java.lang.LinkageError: Failed to link org/eclipse/jetty/server/Connector (Module "org.eclipse.jetty:jetty-server:7.5.4.v20111024" from Ceylon ModuleLoader: RootRepositoryManager: FileContentStore: /home/stephane/.ceylon/cache) which it then wraps into a ClassNotFoundException which is discarded by the caller ModulesClassLoader.LocalLoader$1.loadClassLocal().

I've no idea how to fix this, but we want this exception propagated rather than swallowed, we're losing info here this is not normal. Perhaps newer versions of JBoss modules don't have that behaviour, but I've no idea.

Owner

FroMage commented Apr 3, 2013

Oh, and the LinkageError is caused by java.lang.NoClassDefFoundError: org/eclipse/jetty/util/component/LifeCycle

Owner

FroMage commented Apr 3, 2013

Which is defined in jetty-util but is not imported by jetty-server. How can we even make that work?

Owner

FroMage commented Apr 3, 2013

Well, apparently it depends on it but indirectly.

FroMage added a commit that referenced this issue Apr 3, 2013

Owner

FroMage commented Apr 3, 2013

I pushed the test.

Member

alesj commented Apr 3, 2013

OK, will have a look asap.

Owner

FroMage commented Apr 3, 2013

So can we reasonably assume that Maven artifacts even care about their dependencies at all? This seems to be in favour of @quintesse's opinion that we should make every maven module work in a shared "flat" class loader to mimic their execution on a flat classpath.

Member

alesj commented Apr 3, 2013

They should care.
And even if they don't, we care,
as we need to make sure it runs modular.
e.g. JBossAS was faced with similar issue

As, imo, not doing that, would lead to more issues;
e.g. multiple same classes from diff classloaders, bloated modules, ...

Owner

FroMage commented Apr 3, 2013

They should, but we can't make the world rewrite their POMs, we need to find a way to either detect and fix broken POMs, or flatten it so that it just works, or allow the users to override POMs. We can point fingers at invalid POMs but that won't make them go away, we have to live with them and make them work.

In the case of Camel, that dependency is still not added in the latest version (if it's still required), though I guess we could tell them, but that won't fix other Maven modules.

Member

alesj commented Apr 3, 2013

What's the issue there?
I'm sure there is a reason on why that dependency is missing / is optional.

Owner

FroMage commented Apr 3, 2013

It's not optional, it's just missing: http://repo1.maven.org/maven2/org/eclipse/jetty/jetty-server/7.5.4.v20111024/jetty-server-7.5.4.v20111024.pom

How do I know why they didn't add it?

Owner

quintesse commented Apr 3, 2013

Personally I don't think we should try to fix broken POMs, it's a "feature" of Maven and not something we can do anything about. All the problems mentioned are prob lems of Maven and something that we can accept when importing Maven modules. If in a project you encounter that Maven causes problems we always have the possibility to import the artifacts into Herd and do it right. I don't think we should add a whole bunch of complex heuristics trying to fix things that might not even be fixable.

What's the issue there?

The issue is that Maven uses a flat class path, so I'm pretty sure there are artifacts out there that depend on the fact that they can just access information from other artifacts without having defined any dependencies. They could even be dynamically loading them so scanning the code wouldn't give us any information to try to "fix" things.

That's why I suggested that if Maven uses a flat class path for all its artifacts so should we (for Maven dependencies!) for god knows what kind of things we would be breaking by imposing hierarchical class loading on a system that wasn't designed to support it.

Owner

gavinking commented Apr 3, 2013

That's why I suggested that if Maven uses a flat class path for all its artifacts so should we (for Maven dependencies!) for god knows what kind of things we would be breaking by imposing hierarchical class loading on a system that wasn't designed to support it.

This was my feeling. Maven is broken. Let's not waste time trying to fix it.

Member

alesj commented Apr 3, 2013

It's not optional, it's just missing.

Weird.
I would then expect this to be looked-up via reflection.
And depending on its existence, do something about it.
At least, that's how I do things when additional behavior is optional.

Member

alesj commented Apr 3, 2013

The issue is that Maven uses a flat class path

If it would be flat for 1st level deps, then it would be OK.
The problem is (probably) when some real dependency comes in from 2nd level / transitivity.
Which is just pure bad mvn usage.
Atm, I'm blank with ideas on how to properly handle this.
But, imo, the "flat class path for all its artifacts so should we" is not the right way.
(@dmlloyd - any idea?)

Owner

quintesse commented Apr 3, 2013

And depending on its existence, do something about it.

Sure, but that means we can't fix this.

But if we would handle all Maven dependencies in a single class loader we could just tell people to add an extra dependency on the missing artifact (even if the project doesn't directly use it) and things would work, right? That way at least we have a work-around.

But, imo, the "flat class path for all its artifacts so should we" is not the right way.

Why not? I think we're in for a world of hurt if we try to impose our rules on Maven that wasn't designed with this in mind. (But I'll gladly be proven wrong)

Member

davidfestal commented Apr 3, 2013

But if we would handle all Maven dependencies in a single class loader we
could just tell people to add an extra dependency on the missing artifact
(even if the project doesn't directly use it) and things would work, right?
That way at least we have a work-around.

Yes ! This way we would be able to define wrapper libraries with a module
file that explicitely defines those problematic dependencies and it would
work, without having to import all the transitive deps into herd.
I'd like to do so for the various camel components along with my cameleon
module, but this bug prevents it to run outside Eclipse.

Owner

quintesse commented Apr 12, 2013

I'll recap my point of view on this (also after reading Stef's blog item on modules):

I think treating Maven dependencies outside of the JBoss Module system would be fine. Just add them to the global JVM class path.

Why? Because I don't think the other proposals are viable options. Byte-code analysis is very complex and won't solve all of the problems (eg dynamic class loading), and with user-overridable POMs we pass the problem on to the developer who might not have a clue what is wrong and how to solve it.

Owner

FroMage commented Jun 18, 2013

From other discussions around this subject, I'm starting to wonder if that's enough to make a flat classpath for anything Maven.

Perhaps we'll need a way to override other module dependencies, especially if we start to have conflicting versions in that flat classpath, because I'm not entirely sure we can produce the same classpath ordering that Maven would produce, and we may end up with the wrong version of a duplicate module first rather than last.

Also it's not clear how we can solve the use-case that some Maven modules will attempt to scan the classpath and autoregister/instantiate certain types, based on annotations, like JAX-RS, JPA or CDI. If these Maven modules can't see our Ceylon modules (because those sit outside the Maven CL) then we may need something more.

Owner

FroMage commented Jun 18, 2013

BTW I predict that this issue will become the #1 interop issue for our users. It has already been noticed and we've no workaround so far.

Owner

quintesse commented Jun 18, 2013

Possibly yes, but I still think we need outside help with this.

Although maybe if we do ceylon/ceylon-module-resolver#59 we could lift the handling of Maven modules outside of the JBoss Modules and add them as a flat class path before it. But I'm not even sure if that doable. So discussing this with someone who knows all about JBoss Modules would be nice.

Owner

FroMage commented Jun 18, 2013

Yes but is that going to be enough to have a flat classpath for them, that's my question.

I'm starting to wonder if we don't need something within the notion of assemblies that @gavinking suggested where we can specify a file that can tweak all the module dependencies, but adding/removing/editing them for every module used by the assembly. Note that we could pass it as command-line argument to the ceylon tools for compilation and running, since that's the closest thing we can call an assembly at the moment.

Owner

quintesse commented Jun 18, 2013

Yes but is that going to be enough to have a flat classpath for them, that's my question.

Yeah, I read your first messgae after your second one ;)

So, first of all, wouldn't Aether help us maintain the correct loading order? If it can't that would definitely suck yes.

The part about scanning the classpath makes my head ache ><
Is that even possible in the JBoss Modules universe? Wouldn't that break the encapsulation it's trying to enforce?

Tweaking: I don't know, it all seems very dependant on the programmer knowing exactly what they're doing, which defaults the purpose of having a nice module system that suppsedly knows how to figure it out by itself. Now for very special cases, sure, but this issue right here for example should just work without any tweaking, right?

@FroMage FroMage referenced this issue in eclipse/ceylon-ide-eclipse Aug 6, 2013

Open

Do not include transitive dependencies to the classpath #579

Owner

FroMage commented Sep 16, 2013

All Maven issues move to 1.0

Owner

FroMage commented Oct 21, 2013

This is a CMR issue: ceylon/ceylon-module-resolver#81

@FroMage FroMage closed this Oct 21, 2013

@CeylonMigrationBot CeylonMigrationBot referenced this issue in eclipse/ceylon Nov 14, 2015

Closed

Runtime ClassNotFoundError #4830

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment