How to generate a method call graph of a Jar file directly?

[ghost]

I'm trying to generate a method call graph for a jar file using a programmatic way.

Here is the main code:

public static void main(String[] args) {

 String mainclass = "com.ouc.Example";

 //set classpath
 String javapath = System.getProperty("java.class.path");
 String jredir = System.getProperty("java.home") + "\\lib\\jrt-fs.jar";
 String path = javapath + File.pathSeparator + jredir;

 Scene.v().setSootClassPath(path);

 //add an intra-procedural analysis phase to Soot
 TestCallGraphSootJar_2 analysis = new TestCallGraphSootJar_2();
 PackManager.v().getPack("wjtp").add(new Transform("wjtp.TestSootCallGraph", analysis));

 excludeJDKLibrary();
 Options.v().set_process_dir(
 Arrays.asList("C:\\Users\\username\\ll1\\src\\test\\java\\com\\demo\\dir"));

 //whole program analysis
 Options.v().set_whole_program(true);

 //load and set the main class
 Options.v().set_app(true);
 SootClass appclass = Scene.v().loadClassAndSupport(mainclass);
 Scene.v().setMainClass(appclass);
 Scene.v().loadNecessaryClasses();

 enableSparkCallGraph();

 //start working
 PackManager.v().runPacks();
}

Note

1. “C:\Users\username\ll1\src\test\java\com\demo\dir” is the directory of all .classes files unzipped from the jar file. Because I don't get information about how to directly using the jar file.
2. The above directory doesn't contain the libs that are referenced by my jar.
3. Soot 4.1.0, JDK 14, and Windows 10.

Exception

The statement Scene.v().loadNecessaryClasses(); throw the following exception:

Exception in thread "main" java.lang.RuntimeException: None of the basic classes could be loaded! Check your Soot class path!
	at soot.Scene.loadBasicClasses(Scene.java:1718)
	at soot.Scene.loadNecessaryClasses(Scene.java:1807)
	at com.ouc.TestCallGraphSootJar_2.main(TestCallGraphSootJar_2.java:49)

I have searched a lot, but didn't get useful information. So need your help. Thanks!

[mbenz89]

Hi! Your code looks pretty good from a first glimpse.

Soot generally needs some basic classes from the JDK to work (like Object, Exception, etc.). You can find a listing here: https://github.com/Sable/soot/blob/master/src/main/java/soot/Scene.java#L1656

Here is the place your error is emitted, caused by no basic class being loaded: https://github.com/Sable/soot/blob/master/src/main/java/soot/Scene.java#L1754

I think the issue might really be your classpath:

 String javapath = System.getProperty("java.class.path");
 String jredir = System.getProperty("java.home") + "\\lib\\jrt-fs.jar";
 String path = javapath + File.pathSeparator + jredir;

You are adding the jrt-fs.jar to the class path. But the basic jre classes are contained in jrt.jar. Please try to add this manually or use the -preprend-classpath option to prepend your classpath with the one for execution Soot itself.

[ghost]

@mbenz89 Thank you for your prompt reply. I switch to the JDK 8. And now it works. But there's another question: How can I set to read the jar file by Soot？
The above code throws an exception:

Exception in thread "main" java.lang.RuntimeException: Main-class has no main method!
	at soot.Scene.setMainClass(Scene.java:187)
	at com.ouc.TestCallGraphSootJar_2.main(TestCallGraphSootJar_2.java:48)

I think that it may be caused by the wrong .class files path. I extract all .classes files to a directory and remain the original package structure. Then I use the Options.v().set_process_dir() to set the .class files path. It didn't work, so is there another suggestion? Thanks!

[mbenz89]

Does the class "com.ouc.Example"; contain a main method? Since you set this class as main class, Soot expects it to have a main method (public static void main(...)).

Also, you should load all classes first (especially the basic classes) before you force Soot to load a specific class. Set your main method afterward. Make sure the class containing the main method is on your classpath and then replace

 SootClass appclass = Scene.v().loadClassAndSupport(mainclass);
 Scene.v().setMainClass(appclass);
 Scene.v().loadNecessaryClasses();

by

 Scene.v().loadNecessaryClasses();
 SootClass appclass = Scene.v().getSootClass(mainclass);
 Scene.v().setMainClass(appclass);

Also, are you sure you want to run Soot in app mode? AFAIK this will at least lead to all classes being seen as application classes, i.e. there is no distinguishment between libraries on the classpath and classes on the process dir (see Scene.java:1877).

[ghost]

@mbenz89 Thank you for your kind reply to my question. Unfortunately, it didn't work. The class com.ouc.Example does contain the main method, because I can get its CG using its java source file. Now the code is the following:

public static void main(String[] args) {

	String mainclass = "com.ouc.Example";

	//set classpath
	String javapath = System.getProperty("java.class.path");
	String jredir = System.getProperty("java.home") + "\\lib\\rt.jar";
	String path = javapath + File.pathSeparator + jredir;

	Scene.v().setSootClassPath(path);

	//add an intra-procedural analysis phase to Soot
	TestCallGraphSootJar_2 analysis = new TestCallGraphSootJar_2();
	PackManager.v().getPack("wjtp").add(new Transform("wjtp.TestSootCallGraph", analysis));

	excludeJDKLibrary();

	Options.v().set_process_dir(Arrays.asList("C:\\Users\\username\\ll1\\src\\test\\java\\com\\demo\\dir"));

	//whole program analysis
	Options.v().set_whole_program(true);

	//load and set main class
	Scene.v().loadNecessaryClasses();
	SootClass appclass = Scene.v().getSootClass(mainclass);
	Scene.v().setMainClass(appclass);

	enableSparkCallGraph();

	//start working
	PackManager.v().runPacks();
}

The directory structure is shown in the figure:

[mbenz89]

You're welcome.

I cannot see any problems with youe code. Could you send me the Example.class file and the code of excludeJDKLibrary(); so I can quickly try to reproduce the error?

[ghost]

code and class files.zip

@mbenz89 Thank you for your attention, and sorry for my late reply.
I modified the directory location and reran the program. It works for the "com.ouc.Example" demo.
Then I continued to analyze a jar file called SpotBugs, which is my target. The complete code and class files are attached. Unfortunately, it didn't work. When I debug it, I found the methodList in appclass is null.

[sscui]

Hello, I also want to know how to use soot to generate the call graph of the jar file, and I can use the same command to call the class in rt.jar, but for other jars it shows impossible't find class: android.content.Context ( is your soot-class-path set properly?) (command line) couldn't find class: com.baidu.location.LocationClient (is your soot-class-path set properly?) (in the program) Thank you for your help as soon as possible Solve my doubts @mbenz89
some screenshots are as follows：

（error in the program）

（error in command line）

(class in rt.jar can run normally)

[mbenz89]

@sscui Without any setup, I cannot help. Generally speaking, Soot has a dedicated Android component: FlowDroid. I would suggest asking your question with a bit more information on FlowDroid's issue board.

[mbenz89]

Then I continued to analyze a jar file called SpotBugs, which is my target. The complete code and class files are attached. Unfortunately, it didn't work. When I debug it, I found the methodList in appclass is null.

@liuyuanOUC What is appclass what are your excludes?

[sscui]

@sscui Without any setup, I cannot help. Generally speaking, Soot has a dedicated Android component: FlowDroid. I would suggest asking your question with a bit more information on FlowDroid's issue board.

@mbenz89 Thank you for your reply; my current purpose is just to analyze the jar packages in some SDKs that Android needs to use instand of apk. Should I use FlowDroid? If yes, I will try to learn the implementation of FlowDroid firstly;
Here are some of my settings (PS: rt.jar and BaiduLBS_Android.jar are in the folder src / libs )

[ghost]

@mbenz89 Many thanks.

The appclass is in this statement SootClass appclass = Scene.v().getSootClass(mainclass);.

I found this statement doesn't load the mainclass correctly because the methodclass item of appclass is empty, as shown in the picture. On the other hand, I'm sure that the mainclass does contain main method and its location is right. So it's beyond my knowledge...

[mbenz89]

MIght the app class be phantom? Could you show your exclude list?

[ghost]

@mbenz89 Thanks for your prompt reply.

private static LinkedList<String> excludeList() {
        if (excludeList == null) {
            excludeList = new LinkedList<String>();

            excludeList.add("java.");
            excludeList.add("javax.");
            excludeList.add("sun.");
            excludeList.add("sunw.");
            excludeList.add("com.sun.");
            excludeList.add("com.ibm.");
            excludeList.add("com.apple.");
            excludeList.add("apple.awt.");
        }
        return excludeList;
    }

The excludedList works well. I don't think it is the cause.

[mbenz89]

@sscui If you do not want to analyze an Android application, you probably do not need to use FlowDroid. It mainly assists with taint analysis and generation of a dummy entry point for call graph generation that will call certain lifecycle methods and system/ui handlers.

You probably just don't have all the dependencies that are used by your target code on the classpath. So either add them (if you need to analyze them as well) or allow phantom classes so that soot just generates empty dummies for referenced classes that are not on the classpath. For further questions, please use another issue instead of hijacking this one!

@liuyuanOUC Strang. Is the class phantom? This also seems to be a classpath issue from a first look. If the class is phantom, I would suggest to make sure that soot can actually find the class, i.e. it is on the classpath.

[ghost]

@mbenz89 Thank you. That's really strange. I will review the classpath and record my future solution here.

[ghost]

@mbenz89 Good afternoon, good news!
I add the directory of the class files into the soot_class_path. And it works.
The code is shown as follows.

public static void main(String[] args) {
        //spotbugs -- testing
        String classesDir = "D:\\wkspace\\seed8\\dir\\spotbugs";
        String mainClass = "edu.umd.cs.findbugs.LaunchAppropriateUI";

        //set classpath
        String jreDir = System.getProperty("java.home") + "\\lib\\jce.jar";
        String jceDir = System.getProperty("java.home") + "\\lib\\rt.jar";
        String path = jreDir + File.pathSeparator + jceDir + File.pathSeparator + classesDir;
        Scene.v().setSootClassPath(path);

        //add an intra-procedural analysis phase to Soot
        TestCallGraphSootJar_3 analysis = new TestCallGraphSootJar_3();
        PackManager.v().getPack("wjtp").add(new Transform("wjtp.TestSootCallGraph", analysis));

        excludeJDKLibrary();

        Options.v().set_process_dir(Arrays.asList(classesDir));
        Options.v().set_whole_program(true);
        //Options.v().set_app(true);

        SootClass appClass = Scene.v().loadClassAndSupport(mainClass);
        Scene.v().setMainClass(appClass);
        Scene.v().loadNecessaryClasses();

        //enableCHACallGraph();
        enableSparkCallGraph();

        PackManager.v().runPacks();
    }

If I replace

 SootClass appclass = Scene.v().loadClassAndSupport(mainclass);
 Scene.v().setMainClass(appclass);
 Scene.v().loadNecessaryClasses();

by

 Scene.v().loadNecessaryClasses();
 SootClass appclass = Scene.v().getSootClass(mainclass);
 Scene.v().setMainClass(appclass);

, the program also works.
Note: The code refers to this article （https://o2lab.github.io/710/p/a1.html ）.

[shoaloak]

Sorry for bumping a closed issue, but this is the only way I could contact @liuyuanOUC.

@liuyuanOUC
What does the excludeJDKLibrary(); function do?