Context
The Jelly comparison document (PR #1304, issue #1301) intended to include a Java call graph comparison using "ACG", but the linked tool (github.com/maccioni/acg) is actually a JavaScript adaptive call graph library — not a Java tool.
For Java, established static call graph tools (WALA, Soot, javacg-static) all require compiled bytecode (.class files or JARs), not raw .java source. The current Java fixture contains only source files with no build system.
Current codegraph Java metrics on the fixture:
- Precision: 100%, Recall: 53% (9/17 edges)
- Missed:
same-file (0/2), static (0/2), class-inheritance (0/3)
What's needed
- Add a
Makefile or build.sh to tests/benchmarks/resolution/fixtures/java/ that compiles the fixture with javac and produces a JAR
- Add a
scripts/compare-javacg.mjs (mirrors compare-jelly.mjs) that:
- Runs
javacg-static (github.com/gousiosg/java-callgraph) on the compiled JAR
- Parses its output format (
M:ClassName:method(args)(I)OtherClass:method)
- Maps to the
source/target name format used in expected-edges.json
- Computes precision/recall vs the ground truth
- Update
docs/benchmarks/RESOLUTION-COMPARISON.md with the Java comparison table
Tool choice rationale
javacg-static is the lightest-weight option: it's a single JAR, reads any compiled JAR, and outputs a simple text call graph. WALA and Soot are heavier and require more setup.
Expected impact
Validates (or challenges) codegraph's 53% Java recall claim with an independent tool. javacg-static uses CHA, so it should handle class-inheritance and interface-dispatched edges that codegraph currently misses.
References
Context
The Jelly comparison document (PR #1304, issue #1301) intended to include a Java call graph comparison using "ACG", but the linked tool (
github.com/maccioni/acg) is actually a JavaScript adaptive call graph library — not a Java tool.For Java, established static call graph tools (WALA, Soot,
javacg-static) all require compiled bytecode (.classfiles or JARs), not raw.javasource. The current Java fixture contains only source files with no build system.Current codegraph Java metrics on the fixture:
same-file(0/2),static(0/2),class-inheritance(0/3)What's needed
Makefileorbuild.shtotests/benchmarks/resolution/fixtures/java/that compiles the fixture withjavacand produces a JARscripts/compare-javacg.mjs(mirrorscompare-jelly.mjs) that:javacg-static(github.com/gousiosg/java-callgraph) on the compiled JARM:ClassName:method(args)(I)OtherClass:method)source/target nameformat used inexpected-edges.jsondocs/benchmarks/RESOLUTION-COMPARISON.mdwith the Java comparison tableTool choice rationale
javacg-staticis the lightest-weight option: it's a single JAR, reads any compiled JAR, and outputs a simple text call graph. WALA and Soot are heavier and require more setup.Expected impact
Validates (or challenges) codegraph's 53% Java recall claim with an independent tool. javacg-static uses CHA, so it should handle
class-inheritanceandinterface-dispatchededges that codegraph currently misses.References
docs/benchmarks/RESOLUTION-COMPARISON.md(PR research(bench): Jelly vs codegraph call resolution comparison on JS/TS fixtures #1304)tests/benchmarks/resolution/fixtures/java/expected-edges.jsonscripts/compare-jelly.mjs