Nullpointerexception in JDTJava2CAstTranslator #99

edwardmp · 2016-04-29T16:02:00Z

Hello,

I'm trying to create a call graph but encounter a null pointer exception.
Depending on the java source I use as input the exception is thrown from different visitors, but it always involves "n.resolveMethodBinding()".

resolveMethodBinding returns null because it uses the class BindingResolver's implementation (org.eclipse.jdt.core.dom package) (line 194):

    IMethodBinding resolveMethod(MethodInvocation method) {
        return null;
    }

It is likely that DefaultBindingResolver should be used instead of BindingResolver's which features an empty implementation for each method.

The exact same issue was described by someone else here: https://sourceforge.net/p/wala/mailman/message/34746738/

Hope you have any clue how to solve this issue, as I'm using this tool for research done with regard to my thesis.

Best regards,
Edward

The text was updated successfully, but these errors were encountered:

msridhar · 2016-04-29T19:19:14Z

I don't know this code, nor do I understand Eclipse's AST data structures. @juliandolby any thoughts?

@edwardmp do you see the same stack as in https://sourceforge.net/p/wala/mailman/message/34746738/ ?

edwardmp · 2016-04-29T19:55:19Z

Thanks for the quick response. The stack changes a bit depending on which program I try to analyze, but resolveMethodBinding is the method being called which causes the nullpointerexception, all in invarious visitor methods in that class.

Example trace:

Exception in thread "main" java.lang.NullPointerException
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visit(JDTJava2CAstTranslator.java:1480)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visitNode(JDTJava2CAstTranslator.java:2901)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visit(JDTJava2CAstTranslator.java:1462)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visitNode(JDTJava2CAstTranslator.java:2887)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visitNodeOrNodes(JDTJava2CAstTranslator.java:2959)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.createBlock(JDTJava2CAstTranslator.java:1275)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visit(JDTJava2CAstTranslator.java:1280)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visitNode(JDTJava2CAstTranslator.java:2861)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visit(JDTJava2CAstTranslator.java:806)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.createClassDeclaration(JDTJava2CAstTranslator.java:485)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visitTypeDecl(JDTJava2CAstTranslator.java:408)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visit(JDTJava2CAstTranslator.java:2828)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.translateToCAst(JDTJava2CAstTranslator.java:261)
    at com.ibm.wala.cast.java.translator.jdt.ejc.EJCSourceModuleTranslator$EjcAstToIR.acceptAST(EJCSourceModuleTranslator.java:92)
    at org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:1029)
    at org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:636)
    at org.eclipse.jdt.core.dom.ASTParser.createASTs(ASTParser.java:990)
    at com.ibm.wala.cast.java.translator.jdt.ejc.EJCSourceModuleTranslator.loadAllSources(EJCSourceModuleTranslator.java:181)
    at com.ibm.wala.cast.java.loader.JavaSourceLoaderImpl.loadAllSources(JavaSourceLoaderImpl.java:505)
    at com.ibm.wala.classLoader.ClassLoaderImpl.init(ClassLoaderImpl.java:529)
    at com.ibm.wala.cast.java.loader.JavaSourceLoaderImpl.init(JavaSourceLoaderImpl.java:516)
    at com.ibm.wala.cast.java.translator.jdt.JDTClassLoaderFactory.makeNewClassLoader(JDTClassLoaderFactory.java:69)
    at com.ibm.wala.classLoader.ClassLoaderFactoryImpl.getLoader(ClassLoaderFactoryImpl.java:69)
    at com.ibm.wala.ipa.cha.ClassHierarchy.<init>(ClassHierarchy.java:231)
    at com.ibm.wala.ipa.cha.ClassHierarchy.<init>(ClassHierarchy.java:177)
    at com.ibm.wala.ipa.cha.ClassHierarchy.make(ClassHierarchy.java:1230)
    at com.ibm.wala.cast.java.client.JavaSourceAnalysisEngine.buildClassHierarchy(JavaSourceAnalysisEngine.java:135)
    at eu.sig.aevas.callgraph.WalaStatic.run(WalaStatic.java:165)
    at eu.sig.aevas.AeVas.main(AeVas.java:16)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)

edwardmp · 2016-04-29T19:56:54Z

Other input program stack trace:

Exception in thread "main" java.lang.NullPointerException
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator$TryCatchContext.<init>(JDTJava2CAstTranslator.java:3150)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visit(JDTJava2CAstTranslator.java:2684)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visitNode(JDTJava2CAstTranslator.java:2939)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visitNodeOrNodes(JDTJava2CAstTranslator.java:2959)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.createBlock(JDTJava2CAstTranslator.java:1275)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visit(JDTJava2CAstTranslator.java:1280)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visitNode(JDTJava2CAstTranslator.java:2861)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visit(JDTJava2CAstTranslator.java:806)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.createClassDeclaration(JDTJava2CAstTranslator.java:485)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visitTypeDecl(JDTJava2CAstTranslator.java:408)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visit(JDTJava2CAstTranslator.java:2828)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.translateToCAst(JDTJava2CAstTranslator.java:261)
    at com.ibm.wala.cast.java.translator.jdt.ejc.EJCSourceModuleTranslator$EjcAstToIR.acceptAST(EJCSourceModuleTranslator.java:92)
    at org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:1029)
    at org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:636)
    at org.eclipse.jdt.core.dom.ASTParser.createASTs(ASTParser.java:990)
    at com.ibm.wala.cast.java.translator.jdt.ejc.EJCSourceModuleTranslator.loadAllSources(EJCSourceModuleTranslator.java:181)
    at com.ibm.wala.cast.java.loader.JavaSourceLoaderImpl.loadAllSources(JavaSourceLoaderImpl.java:505)
    at com.ibm.wala.classLoader.ClassLoaderImpl.init(ClassLoaderImpl.java:529)
    at com.ibm.wala.cast.java.loader.JavaSourceLoaderImpl.init(JavaSourceLoaderImpl.java:516)
    at com.ibm.wala.cast.java.translator.jdt.JDTClassLoaderFactory.makeNewClassLoader(JDTClassLoaderFactory.java:69)
    at com.ibm.wala.classLoader.ClassLoaderFactoryImpl.getLoader(ClassLoaderFactoryImpl.java:69)
    at com.ibm.wala.ipa.cha.ClassHierarchy.<init>(ClassHierarchy.java:231)
    at com.ibm.wala.ipa.cha.ClassHierarchy.<init>(ClassHierarchy.java:177)
    at com.ibm.wala.ipa.cha.ClassHierarchy.make(ClassHierarchy.java:1230)
    at com.ibm.wala.cast.java.client.JavaSourceAnalysisEngine.buildClassHierarchy(JavaSourceAnalysisEngine.java:135)
    at eu.sig.aevas.callgraph.WalaStatic.run(WalaStatic.java:165)
    at eu.sig.aevas.AeVas.main(AeVas.java:16)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)

edwardmp · 2016-04-29T19:59:59Z

By the way I use 1.3.9-SNAPSHOT versions of all WALA libraries, since I could not install some through maven (com.ibm.wala.cast.java and com.ibm.wala.ide.jdt)

edwardmp · 2016-05-01T16:37:56Z

Related:
http://stackoverflow.com/questions/2017945/bindings-not-resolving-with-ast-processing-in-eclipse/5803778#5803778
http://stackoverflow.com/questions/2631981/why-does-resolvebinding-return-null-even-though-i-setresolvebindingstrue-on

edwardmp · 2016-05-01T20:15:22Z

I think I know why the issue arose:
I had to add jars of all libraries to by calling addSystemModule of class JavaSourceAnalysisEngine.
I needed to add rt.jar when my source files used no external libraries, and if they did .jars with all external libraries.

msridhar · 2016-05-01T21:01:26Z

Thanks for digging in! We still shouldn't crash for missing jars. We should add some catch blocks that log a warning.

edwardmp · 2016-05-02T07:45:49Z

Yes I agree that a more helpful message would be convenient.

Also @msridhar, is there any way com.ibm.wala.cast.java and com.ibm.wala.ide.jdt could be made available in the maven repository?

Finally, if I add my source files through addSourceModule why doesn't the condition is this codeblock ever to false? Any clue?

if (!analysisScope.isApplicationLoader(c.getClassLoader())) {
                continue;
 }

I see it is a source loader and not an application loader. It works if I try to get the parent of the source loader, which is a application loader. But isn't there a better way?

edwardmp · 2016-05-03T16:09:02Z

@msridhar ideally, in my case, I would like to continue building the dependency graph when a lib jar is missing, just behave as if that library would have been added to the exclusions file. Is this possible?

msridhar · 2016-05-03T17:25:12Z

@edwardmp that's what I was suggesting above. A hackish fix would be to add null checks and then just soldier on without the relevant method / field. WALA is generally robust to missing classes, etc., it's just this frontend that is falling down. I am very busy for the next few days, but I can try to take a look after that. Or, if you could attempt to throw together a patch and make a pull request, I can give comments.

Re: Maven Central, the problem is that these projects depend on Eclipse, and I'm not sure of the right way to specify such dependencies in a Maven jar. There must be some way to do it (after all, we can build WALA with Maven), I just don't know how to do it. Again, if someone could do some legwork here that would be helpful.

Re: addSourceModule, I don't know the answer. @juliandolby any thoughts?

edwardmp · 2016-05-09T12:41:08Z

@msridhar OK. Regarding the availability in the Maven repository, I am trying to add the missing projects but I am not that experienced with Maven and the setup is a bit complex, will definitely dive deeper into this later.

In the meantime, I want to use WALA in a production environment at the company I do an internship at, but they refuse to let me use the current 1.3.9-SNAPSHOT. Is there any way you could release 1.3.9 anytime soon?

Thanks in advance,

msridhar · 2016-05-09T13:12:23Z

I'm fine with cutting a new release. Let me check with Julian. --Manu

On Mon, May 9, 2016 at 9:41 PM Edward Poot notifications@github.com wrote:

@msridhar https://github.com/msridhar OK. Regarding the availability in
the Maven repository, I am trying to add the missing projects but I am not
that experienced with Maven and the setup is a bit complex, will definitely
dive deeper into this later.

In the meantime, I want to use WALA in a production environment at the
company I do an internship at, but they refuse to let me use the current
1.3.9-SNAPSHOT. Is there any way you could release 1.3.9 anytime soon?

Thanks in advance,

—
You are receiving this because you were mentioned.

Reply to this email directly or view it on GitHub
#99 (comment)

msridhar · 2016-05-10T14:03:39Z

I have tagged release 1.3.9 in the git repository (R_1.3.9). Still working on pushing new jars to Maven Central, should be done in the next day or so.

edwardmp · 2016-05-10T14:04:30Z

Great, thanks a lot.

edwardmp · 2016-05-18T17:28:31Z

Hi @msridhar,

Did you have any opportunity yet to change how missing library jars are handled?
This would be a great case for the use of Optionals in Java 8, although personally I have no experience in using them.

msridhar · 2016-05-18T17:32:35Z

Haven't had time to look at this yet, and not sure when I will. At the
least, anyone trying to fix will need a test case for reproducing the
error, along with instructions on how to reproduce.

On Wed, May 18, 2016, 10:28 AM Edward Poot notifications@github.com wrote:

Hi @msridhar https://github.com/msridhar,

Did you have any opportunity yet to change how missing library jars are
handled?
This would be a great case for the use of Optionals in Java 8, although
personally I have no experience in using them.

—
You are receiving this because you were mentioned.

Reply to this email directly or view it on GitHub
#99 (comment)

edwardmp · 2016-05-18T17:34:11Z

OK. I can certainly temporary fix it, although doing it in a proper way requires much knowhow of how the library operates. Tomorrow I will put some example code in a repository so it can be easily reproduced.

edwardmp · 2016-05-26T16:47:52Z

Hi @msridhar

Finally had some time to provide an example displaying the issue.
This is the tiny system I want to create a callgraph for: https://github.com/edwardmp/wala-issue-example
This tiny example depends on the Netty library.

Now I want to only use the source files to generate the callpgrah, using the Eclipse Java frontend.

This is the code used to generate the callgraph:
https://github.com/edwardmp/wala-callgraph-generation-issue-example

The code in main first tries to generate the graph without adding the Netty.jar as a system module. Run it and you will see the null pointer exception. The commented out code in main shows that if you do run it and specify the jar, everything works OK.

Ideally, WALA should log an error but not throw an exception and keep generating the call graph but just exclude calls to that library.

Don't forget to change the hardcoded path in main to point to the the download location of the first repository I mentioned (hardcoded path in my example is /Users/Edward/Desktop/wala-issue-example).

Hope this helps!

edwardmp · 2016-05-30T07:56:08Z

@msridhar is it OK if a open a pull request in which some null checks are added?

msridhar · 2016-05-30T13:28:32Z

Sure, that would be greatly appreciated! By the way, as you may have
noticed, I've done some refactoring on the master branch. From now on, you
should be able to use the wala.cast.java and wala.cast.java.ecj
projects using Maven Central jars. If you'd like to test it out, I can
create some snapshot jars; let me know. If this is what you needed, we can
cut a new release for it.

--Manu

On Mon, May 30, 2016 at 12:56 AM Edward Poot notifications@github.com
wrote:

@msridhar https://github.com/msridhar is it OK if a open a pull request
in which some null checks are added?

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
#99 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AALyUSZMLwgxCnwR1RpvMDMBo_9vVb-aks5qGpgZgaJpZM4IS85m
.

edwardmp · 2016-05-30T15:35:22Z

@msridhar
That is very convenient, but what is more important is the problem mentioned above leading to a NPE.

I've been pulling my hair out the last few hours trying to run WALA on for example PMD (https://github.com/pmd/pmd, ran on pmd-core dir), but this again leads to a null pointer exception. But in this case, the reason doesn't seem to be related to missing jars from dependencies.

Caused by: java.lang.NullPointerException
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.createClassDeclaration(JDTJava2CAstTranslator.java:473)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visitTypeDecl(JDTJava2CAstTranslator.java:408)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visit(JDTJava2CAstTranslator.java:2828)
    at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.translateToCAst(JDTJava2CAstTranslator.java:261)
    at com.ibm.wala.cast.java.translator.jdt.ejc.EJCSourceModuleTranslator$EjcAstToIR.acceptAST(EJCSourceModuleTranslator.java:92)
    at org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:1029)
    at org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:636)
    at org.eclipse.jdt.core.dom.ASTParser.createASTs(ASTParser.java:990)
    at com.ibm.wala.cast.java.translator.jdt.ejc.EJCSourceModuleTranslator.loadAllSources(EJCSourceModuleTranslator.java:181)
    at com.ibm.wala.cast.java.loader.JavaSourceLoaderImpl.loadAllSources(JavaSourceLoaderImpl.java:505)
    at com.ibm.wala.classLoader.ClassLoaderImpl.init(ClassLoaderImpl.java:529)
    at com.ibm.wala.cast.java.loader.JavaSourceLoaderImpl.init(JavaSourceLoaderImpl.java:516)
    at com.ibm.wala.cast.java.translator.jdt.JDTClassLoaderFactory.makeNewClassLoader(JDTClassLoaderFactory.java:69)
    at com.ibm.wala.classLoader.ClassLoaderFactoryImpl.getLoader(ClassLoaderFactoryImpl.java:69)
    at com.ibm.wala.ipa.cha.ClassHierarchy.<init>(ClassHierarchy.java:231)
    at com.ibm.wala.ipa.cha.ClassHierarchy.<init>(ClassHierarchy.java:177)
    at com.ibm.wala.ipa.cha.ClassHierarchy.make(ClassHierarchy.java:1230)
    at com.ibm.wala.cast.java.client.JavaSourceAnalysisEngine.buildClassHierarchy(JavaSourceAnalysisEngine.java:135)

This is caused because IVariableBinding fieldBinding = fieldFrag.resolveBinding(); has a value of null, thus it is again related to the resolving of bindings. It could be that the issue I opened this issue for isn't the real issue but just a symptom. Something is seriously wrong in resolving bindings.

Kind of frustrating as I really need to generate a call graph of various systems for research related to my thesis and I don't have much time to dive into this.

msridhar · 2016-05-30T15:39:55Z

@edwardmp do you really need to use the Java source front end? All the problems here are in the source front end. The Java bytecode front end is much more robust.

edwardmp · 2016-05-30T15:41:20Z

@msridhar
Unfortunately, the company I conduct my research at only has certain clients' systems source code available and (usually) not any binaries. I tried a few other call graph libraries before using WALA but those wouldn't work either.

msridhar · 2016-05-30T16:29:51Z

Ok. What makes you think this is not due to missing jars? In any case, as a patch-up fix, my approach would be to add null checks around any relevant calls to resolveBindings(), and then "bail out" if the reset is null. E.g., for the stack you just posted, we just wouldn't add the FieldEntity on the next line. If you submit a PR that adds null checks of this form, I'll take a quick look at it and then merge.

edwardmp · 2016-05-30T16:32:09Z

@msridhar
Well basically because I have checked that I have jars of all dependencies. Also, in the PMD case I deleted most source files and traced it to 1 specific file (that only had a dependency on a subdir).

OK I will try to add some null checks. With bailing out you mean just skipping the next line or returning from the method? Thanks for your cooperation! much appreciated

msridhar · 2016-05-30T16:38:27Z

By bailing out I generally mean for whatever operation the binding is being used for (typically adding something to a class or class hierarchy), don't do that operation. But if you're a little over-aggressive in bailing out, that's fine for now (better than crashing).

edwardmp · 2016-05-31T15:14:32Z

@msridhar
spent the last few hours adding checks but this is not really a solution; sometimes you can skip a part of a method but other times you can only return null from that method in case the binding is null, which just propagates the NPE to another place.. Digging in to find the real reason the bindings are not set but no luck so far

edwardmp · 2016-05-31T17:48:43Z

@msridhar

So far it boils down to this:
setRecoveryBindings must be set to true when the ASTParser is created.

I have a small test case where a ClassInstanceCreation can not resolve the constructor bindings while all other bindings are actually present.

Example statement: new ArrayList<>(10); does have constructor bindings and new ArrayList<>(); (without a capacity) does have bindings . But when the capacity is dependent upon another list (e.g. new ArrayList<>(otherList.size()); it does not have bindings` Why is still the big question..

edwardmp · 2016-05-31T18:17:39Z

@msridhar
Here is a more complete example:

public abstract class AbstractPropertySource implements PropertySource {
    protected List<PropertyDescriptor<?>> copyPropertyDescriptors() {
        List<PropertyDescriptor<?>> b = new ArrayList<>(); // works, resolveConstructorBindings() does not return null
        List<PropertyDescriptor<?>> c = new ArrayList<>(1); // works fine
        List<PropertyDescriptor<?>> copy = new ArrayList<>(b.size()); // null pointer exception because resolveConstructorBinding() returns null
        copy.addAll(propertyDescriptors);
        return copy;
    }
}

Maybe its a bug in the JDT library?

msridhar · 2016-05-31T18:58:15Z

Thanks for digging in! So is adding a call to setRecoveryBindings somewhere
sufficient to fix this?

On Tue, May 31, 2016, 11:17 AM Edward Poot notifications@github.com wrote:

@msridhar https://github.com/msridhar
Here is a more complete example:

public abstract class AbstractPropertySource implements PropertySource {
protected List<PropertyDescriptor> copyPropertyDescriptors() { List> b = new ArrayList<>(); // works, v() does not return null
List<PropertyDescriptor<?>> copy = new ArrayList<>(b.size()); // null pointer exception because resolveConstructorBinding() returns null
copy.addAll(propertyDescriptors);
return copy;
}
}

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#99 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AALyUeX46Vhb3eAui2Z3JGjwJxeFgJM-ks5qHHtDgaJpZM4IS85m
.

edwardmp · 2016-05-31T19:01:12Z

@msridhar

In the code I used to reproduce it at some point it had to setRecoveryBindings to true, but in the ultimate code I have changing it for this example does not have immediate impact.

So no, unfortunately that's not enough. In the example above there are cases where resolveConstructorBindings() still returns null while it should. Trying if I can add some null checks for that, but bailing out there will probably cause issues somewhere else..

Most likely, its a bug in the JDT core. Last version 3.10 was released in early 2015, seems the project is inactive maybe?

I'm going to try filing a bug for JDT. Odd thing is that the bug reporter is listing versions different than 3.10, seems as if JDT core in the bugtracker refers to another project? Any idea?

I have to say this is driving me nuts, as I don't expect a fix any time soon, which means I can't really use WALA for my research. After having setup 3 previous call graph libraries that didn't work this is kind of frustrating.

msridhar · 2016-05-31T20:36:00Z

@edwardmp I'm really sorry this doesn't seem to be working out. I am a bit unsure about the version numbering of JDT Core. I thought we are pulling in the latest version, but I'm not sure. Let me look into it.

edwardmp · 2016-05-31T20:50:44Z

Thanks, of course I don't blame you or any other WALA maintainer. It's just annoying it eventually worked fine for a sample project but not any other it seems.

I've been reading up on alternatives for call graph generation. Seems you can use some JDT stuff for that, using the CallHierarchy. Its already in the WALA code somewhere: https://github.com/wala/WALA/blob/e87b2e5d843796246a856be65e7cb7b8251c4a47/com.ibm.wala.ide.jsdt/source/com/ibm/wala/ide/util/JsdtUtil.java

Do you happen to know more about this? Maybe its an alternative approach. See also http://www.programcreek.com/2011/07/find-all-callers-of-a-method/

msridhar · 2016-05-31T21:30:09Z

Yeah, so Eclipse has had this call hierarchy feature for finding callers of a method for a while. I'm guessing you would have to be inside a running Eclipse instance to use this feature, i.e., it'll be hard to use just from the command line. But I have no personal experience trying to programmatically invoke the call hierarchy. ( @juliandolby wrote that Jsdt code; he might be able to chime in with more tips.)

edwardmp · 2016-05-31T21:35:56Z

@msridhar

Thanks. Kind of confused as if WALA actually uses the latest JDK core version? The artifact id is different than the one used though: https://github.com/eclipse/eclipse.jdt.core/blob/master/pom.xml

msridhar · 2016-05-31T21:41:49Z

Yeah, it's a bit of a mess. We have the Eclipse plugin build system which uses Tycho, and will bring in some version of JDT Core; I think that's Eclipse Luna, which is not the latest. Then, I've been writing these mvncentral.xml files which try to make it easier to use WALA outside of Eclipse, and that's what is pulling in 3.10. Is it possible maybe for you to plug in a more recent version and see if that fixes things? If so, we can try to figure out how to update what WALA pulls in.

edwardmp · 2016-05-31T21:45:24Z

Yes thats my intention. It is not unthinkable that if its a bug, it may be fixed in newer versions.
Its actually hard to find a JAR with a newer release but I found org.eclipse.jdt.core.source_3.11.2.v20160128-0629.jar in https://www.eclipse.org/downloads/download.php?file=/eclipse/downloads/drops4/R-4.5.2-201602121500/org.eclipse.jdt-4.5.2.zip (under plugins folder) which seems to contain the necessary files.. experimenting if it might be compatible out of the box.

msridhar · 2016-05-31T23:18:17Z

Ok, let us know how it goes.

edwardmp · 2016-06-01T13:47:47Z

@msridhar
OK using so org.eclipse.jdt.core_3.11.2 which is slightly newer still leads to an exception, but it does progress a bit further. Furthermore the exception is not a NPE but Assertions.UNREACHABLE("Couldn't find default ctor");

I tried another call graph library which seemed to work, only to discover that it has issues with handling java 8 source code, so I'm pretty desperate for this to work.

edwardmp · 2016-06-01T14:22:42Z

Nope unfortunately, in the end throws NPE at me again. I think I have to reconsider my research subject since every call graph library I tried had issues some way or another.

msridhar · 2016-06-01T14:51:49Z

@edwardmp in cases where you have all the dependent libraries, could you just compile the source code? Then you could avoid these JDT issues.

edwardmp · 2016-06-01T15:03:55Z

@msridhar If I do it manually, I might be able for some systems but my employer generally does not necessarily have all (e.g. proprietary) dependencies. But the input for my system using WALA is supposed to be dynamic, e.g. when my employer obtains a new system it should just work without manually compiling it.

msridhar · 2016-06-01T17:19:22Z

@edwardmp ok, unfortunately, I don't know of any system that can handle your needs out of the box, unfortunately. Beyond the JDT issues we've been dealing with, you mentioned Java 8 source code above. While JDT supports Java 8, I don't think the WALA source front end supports constructs like lambdas. So that's another issue.

For right now, the best thing I can think of for using WALA would be to stick a try-catch block inside ECJSourceModuleTranslator.acceptAST():

https://github.com/wala/WALA/blob/master/com.ibm.wala.cast.java.ecj/src/com/ibm/wala/cast/java/translator/jdt/ejc/ECJSourceModuleTranslator.java#L90

Could you just catch any NPE thrown by the translateToCAst() call and immediately return? This might not work if translateToCAst() mutates data structures and leaves things in an inconsistent state. But it's the best thing I can think of at this point. Otherwise, unfortunately WALA may just not work for this purpose.

edwardmp · 2016-06-01T17:22:04Z

@msridhar
Thanks, I'll try that as a last resort. Actually this library: https://github.com/Sable/soot does work with source code but only not Java 8 code.. "funny" enough.

Lambda's appeared to work in one small sample project.

edwardmp · 2016-06-02T10:21:24Z

Well, didn't expect much but catching Null Pointer exceptions works in the short term (but get a lot of "unexpected problems" reported but eventually another exception is thrown.
Do note that all missing classes are excluded in the exclusions file.

505. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Lsun/misc/SignalHandler
506. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Lsun/reflect/LangReflectAccess
507. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Lsun/security/krb5/JavaxSecurityAuthKerberosAccess

    at com.ibm.wala.ipa.callgraph.propagation.PropagationCallGraphBuilder.makeCallGraph(PropagationCallGraphBuilder.java:263)
    at com.ibm.wala.client.AbstractAnalysisEngine.buildCallGraph(AbstractAnalysisEngine.java:137)
    at com.ibm.wala.client.AbstractAnalysisEngine.defaultCallGraphBuilder(AbstractAnalysisEngine.java:309)

edwardmp · 2016-06-02T11:01:34Z

In my case, I'm thinking about an alternative. I can use JDT to visit all MethodInvocations. Then I can find method calls to a specific library I'm interested in. Then I can use WALA to trace calls within this library (for those libraries I do have the jars so running on bytecode shouldn't pose problems).

edwardmp · 2016-06-02T16:44:42Z

@msridhar I seems my solution works well. I don't really care about the source code other than that I want to find calls to a specific library. Using ASTParser I obtain those, then I run wala on the library jar.

Great. I do think however that some warnings should be put in place to warn people that the source translation is "experimental/buggy"

msridhar · 2016-06-02T21:39:54Z

Ok, glad you figured something out. We should indeed add a warning in the
documentation. I'll leave this issue open until I do so.

On Thu, Jun 2, 2016, 9:44 AM Edward Poot notifications@github.com wrote:

@msridhar https://github.com/msridhar I seems my solution works well. I
don't really care about the source code other than that I want to find
calls to a specific library. Using ASTParser I obtain those, then I run
wala on the library jar.

Great. I do think however that some warnings should be put in place to
warn people that the source translation is "experimental/buggy"

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#99 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AALyURNBufgE_vS27CKu-c2SioSPRKqjks5qHwh6gaJpZM4IS85m
.

edwardmp · 2016-06-06T11:30:35Z

@msridhar
For one system I encountered the following exception:

[ERROR][2016-06-06 13:23:43][WalaStaticJarInput] Caught exception during callgraph generation: Could not create a entrypoint callsites: 1. [Moderate] class com.ibm.wala.ipa.cha.ClassHierarchy$ClassExclusion : <Application,Lorg/eclipse/jetty/websocket/client/ClientUpgradeRequest> No superclass found for <Application,Lorg/eclipse/jetty/websocket/client/ClientUpgradeRequest> Superclass name Lorg/eclipse/jetty/websocket/api/UpgradeRequest
2. [Moderate] class com.ibm.wala.ipa.cha.ClassHierarchy$ClassExclusion : <Application,Lorg/eclipse/jetty/websocket/client/ClientUpgradeResponse> No superclass found for <Application,Lorg/eclipse/jetty/websocket/client/ClientUpgradeResponse> Superclass name Lorg/eclipse/jetty/websocket/api/UpgradeResponse
3. [Moderate] class com.ibm.wala.ipa.cha.ClassHierarchy$ClassExclusion : <Application,Lorg/eclipse/jetty/websocket/client/WebSocketClient> No superclass found for <Application,Lorg/eclipse/jetty/websocket/client/WebSocketClient> Superclass name Lorg/eclipse/jetty/util/component/ContainerLifeCycle

I noticed this happens when you don't add any entrypoints. But since I check for that before I generate a call graph, in this case that's not the issue. Any idea?

msridhar · 2016-06-06T21:03:16Z

@edwardmp no idea. If it's unrelated to the JDT issue, best to discuss on the mailing list or a separate issue if it's a bug.

rizktouma · 2017-09-15T09:37:38Z

@edwardmp I am facing the same (original) problem that @msridhar posted in this thread. In fact, I am getting exactly the same stacktrace of the null pointer exception.

The problem, surprise surprise, is that resolveMethodBinding() is returning null. It is caused by the invocation of a method in a class in a JAR file included in my project. So far, I have tried the following solutions without success:

Made sure all Java standard libraries (including rt.jar) are included
Updated the Eclipse JDT libraries to the latest version (3.12 released in Feb 2017)
Tried with different versions of Java (7 and 8)
Set the ASTParser properties as suggested here

All other bindings seem to be working fine (even those with invocations of other methods in the same class) and I have no clue why this specific one is failing. Do you have any idea what might be causing it?

qingdujun · 2019-06-03T10:07:28Z

//...

Set the ASTParser properties as suggested here

@rizktouma

About WALA(not ASTViewer), How to get ASTParse and set it?

mattkindy mentioned this issue Jan 28, 2020

How to add source files and what is the entrypoint? #520

Open

msridhar mentioned this issue Apr 9, 2020

Better Wala error message MagpieBridge/IRConverter#1

Open

liblit added the ide Importing, building, and developing WALA in IDEs such as IntelliJ IDEA or Eclipse label Sep 5, 2021

msridhar mentioned this issue Sep 2, 2023

ClassHierarchyFactory.makeWithRoot fails when analyzing source code without dependencies #1314

Open

msridhar mentioned this issue Jul 6, 2024

SourceDirCallGraph.java Exception java.lang.NoClassDefFoundError: IProgressMonitor #1411

Closed

Nullpointerexception in JDTJava2CAstTranslator #99

Nullpointerexception in JDTJava2CAstTranslator #99

Comments

edwardmp commented Apr 29, 2016 • edited Loading

msridhar commented Apr 29, 2016

edwardmp commented Apr 29, 2016 • edited Loading

edwardmp commented Apr 29, 2016 • edited Loading

edwardmp commented Apr 29, 2016

edwardmp commented May 1, 2016

edwardmp commented May 1, 2016 • edited Loading

msridhar commented May 1, 2016

edwardmp commented May 2, 2016 • edited Loading

edwardmp commented May 3, 2016

msridhar commented May 3, 2016

edwardmp commented May 9, 2016

msridhar commented May 9, 2016

msridhar commented May 10, 2016

edwardmp commented May 10, 2016

edwardmp commented May 18, 2016

msridhar commented May 18, 2016

edwardmp commented May 18, 2016

edwardmp commented May 26, 2016

edwardmp commented May 30, 2016

msridhar commented May 30, 2016

edwardmp commented May 30, 2016 • edited Loading

msridhar commented May 30, 2016

edwardmp commented May 30, 2016 • edited Loading

msridhar commented May 30, 2016

edwardmp commented May 30, 2016

msridhar commented May 30, 2016

edwardmp commented May 31, 2016

edwardmp commented May 31, 2016 • edited Loading

edwardmp commented May 31, 2016 • edited Loading

msridhar commented May 31, 2016

edwardmp commented May 31, 2016 • edited Loading

msridhar commented May 31, 2016

edwardmp commented May 31, 2016

msridhar commented May 31, 2016

edwardmp commented May 31, 2016

msridhar commented May 31, 2016

edwardmp commented May 31, 2016

msridhar commented May 31, 2016

edwardmp commented Jun 1, 2016 • edited Loading

edwardmp commented Jun 1, 2016

msridhar commented Jun 1, 2016

edwardmp commented Jun 1, 2016

msridhar commented Jun 1, 2016

edwardmp commented Jun 1, 2016

edwardmp commented Jun 2, 2016 • edited Loading

edwardmp commented Jun 2, 2016 • edited Loading

edwardmp commented Jun 2, 2016

msridhar commented Jun 2, 2016

edwardmp commented Jun 6, 2016

msridhar commented Jun 6, 2016

rizktouma commented Sep 15, 2017 • edited Loading

qingdujun commented Jun 3, 2019

edwardmp commented Apr 29, 2016 •

edited

Loading

edwardmp commented Apr 29, 2016 •

edited

Loading

edwardmp commented Apr 29, 2016 •

edited

Loading

edwardmp commented May 1, 2016 •

edited

Loading

edwardmp commented May 2, 2016 •

edited

Loading

edwardmp commented May 30, 2016 •

edited

Loading

edwardmp commented May 30, 2016 •

edited

Loading

edwardmp commented May 31, 2016 •

edited

Loading

edwardmp commented May 31, 2016 •

edited

Loading

edwardmp commented May 31, 2016 •

edited

Loading

edwardmp commented Jun 1, 2016 •

edited

Loading

edwardmp commented Jun 2, 2016 •

edited

Loading

edwardmp commented Jun 2, 2016 •

edited

Loading

rizktouma commented Sep 15, 2017 •

edited

Loading