Problem in daemon expiration check java.lang.OutOfMemoryError: Java heap space when using javaProject.createOnDemandView().getClasses()

[Dariece]

Hello,
First of all: i'm a soot and sootUp noob. I use sootUp for the first time in a bachelor thesis software project.

For my own gradle plugin I need to check incompatibility from target project classes to resolved runtime dependencies. Actually I try to map all classes of a dependency jar to its filename. Unfortunately dependencies like org.bouncycastle:bcprov-jdk15on cause java.lang.OutOfMemoryError: Java heap space exception, maybe too many classes to load in heap? Is there any other way to solve this problem except using a blacklist for dependencies like these?

Code:

import java.nio.file.Path;
import java.util.List;
import org.eclipse.collections.api.factory.Lists;
import org.eclipse.collections.api.factory.Maps;
import org.eclipse.collections.api.factory.Sets;
import org.eclipse.collections.api.list.ImmutableList;
import org.eclipse.collections.api.map.ImmutableMap;
import org.eclipse.collections.api.map.MutableMap;
import org.eclipse.collections.api.set.ImmutableSet;
import org.gradle.api.GradleException;
import org.gradle.api.logging.Logger;
import org.gradle.api.logging.Logging;
import sootup.core.inputlocation.AnalysisInputLocation;
import sootup.core.model.SourceType;
import sootup.java.bytecode.inputlocation.PathBasedAnalysisInputLocation;
import sootup.java.core.JavaProject;
import sootup.java.core.JavaSootClass;
import sootup.java.core.language.JavaLanguage;

public class SootUtil {

  /**
   * lib that takes a long time to get call-graph or causes java.lang.OutOfMemoryError: Java heap space by getting all classes
   */
  private static final ImmutableSet<String> LONG_TIME_LIB = Sets.immutable.of(
      "org.scala-lang:scala-library", "org.clojure:clojure", "org.bouncycastle:bcprov-jdk15on");

  private static final Logger log = Logging.getLogger(SootUtil.class);

  public static JavaProject configureProject(int javaVersion, Path binaryPath,
      boolean isRootProject) {
    JavaLanguage language = new JavaLanguage(javaVersion);
    AnalysisInputLocation<JavaSootClass> inputLocation = new PathBasedAnalysisInputLocation(
        binaryPath, isRootProject ? SourceType.Application : SourceType.Library);

    JavaProject project = JavaProject.builder(language).addInputLocation(inputLocation).build();
    return project;
  }

  public static ImmutableList<JavaSootClass> getAllProjectClasses(JavaProject javaProject) {
    return Lists.immutable.ofAll(javaProject.createOnDemandView().getClasses())
        .select(javaSootClass -> !javaSootClass.getName().startsWith("classes.java.test"));
  }

  public static ImmutableMap<String, ImmutableList<JavaSootClass>> getJarDependencyClasses(
      List<Path> resolvedJarFiles) {
    MutableMap<String, ImmutableList<JavaSootClass>> dependencyJarClasses = Maps.mutable.empty();

    //TODO get JavaVersion for each jar file
    resolvedJarFiles.forEach(path -> {
      String jarFilename = path.getFileName().toString();
      log.quiet("{}: ", jarFilename);
      ImmutableList<JavaSootClass> jarClasses = tryGetClassesForJavaVersion(path, 17);
      log.quiet("   --> Works!");
      dependencyJarClasses.put(jarFilename, jarClasses);
    });

    return dependencyJarClasses.toImmutable();
  }

  private static ImmutableList<JavaSootClass> tryGetClassesForJavaVersion(Path path,
      int javaVersion) {
    try {
      log.debug("JavaVersion: {}", javaVersion);
      JavaProject dependencyJar = configureProject(javaVersion, path, false);
      return getAllProjectClasses(dependencyJar);
    } catch (IllegalArgumentException e) {
      if (javaVersion >= 8) {
        return tryGetClassesForJavaVersion(path, --javaVersion);
      }
      throw new GradleException(
          String.format("%s: Could not resolve javaVersion for dependency jar %s",
              SootUtil.class.getName(), path), e);
    }
  }
}

Happens when: javaProject.createOnDemandView().getClasses() is called for bcprov-jdk15on-1.69.jar

Exception message is: "Exception in thread "Daemon client event forwarder" java.lang.OutOfMemoryError: Java heap space"

Log:

...
logback-core-1.2.11.jar:
--> Works!
slf4j-api-1.7.36.jar:
--> Works!
log4j-api-2.17.2.jar:
--> Works!
bcutil-jdk15on-1.69.jar:
--> Works!
bcprov-jdk15on-1.69.jar:

Exception in thread "Daemon client event forwarder" java.lang.OutOfMemoryError: Java heap space

Task :checkCompatibility FAILED

FAILURE: Build failed with an exception.

• What went wrong:
  Execution failed for task ':checkCompatibility'.

Java heap space

• Try:

Run with --info or --debug option to get more log output.
Run with --scan to get full insights.

• Exception is:
  org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':checkCompatibility'.
  at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.lambda$executeIfValid$1(ExecuteActionsTaskExecuter.java:142)
  at org.gradle.internal.Try$Failure.ifSuccessfulOrElse(Try.java:282)
  at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:140)
  at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:128)
  at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:77)
  at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
  at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:51)
  at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
  at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:57)
  at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
  at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
  at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
  at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
  at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204)
  at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199)
  at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
  at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
  at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
  at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
  at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53)
  at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73)
  at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52)
  at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:69)
  at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:322)
  at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:309)
  at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:302)
  at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:288)
  at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:462)
  at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:379)
  at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
  at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
  Caused by: java.lang.OutOfMemoryError: Java heap space

[JonasKlauke]

Hi Dariece,
The reason for the exception is the endless loop :) tryGetClassesForJavaVersion will set the java version to 15 and if you catch the execption you will enter the method again and set the version to 15 ......

[Dariece]

Hi @JonasKlauke,
sry for the confusion. The Java 15 Statement wasn't the main reason for the problem, just a test because I thought the wrong java version for the dependency bcprov-jdk15on had been the problem. The heap exception happens at the first try to configure a project.

I think the main cause is, that I tried to collect all classes of all dependencies at once = full heap

BTW. is there any more elegant way to find a fitting javaVersion than try/error or parse the .jar MANIFEST manually?