Skip to content

StringExamples generation exceeds timeout in 2-3 times #1141

New issue
New issue
Closed
#1151
#1144
Closed
StringExamples generation exceeds timeout in 2-3 times#1141
#1151
#1144
Assignees
Domonion
Labels
ctg-bugIssue is a bugstatus-verifiedBug fix is verified
@alisevych

Description

@alisevych
alisevych
opened on Oct 11, 2022 · edited by alisevych
Member

Description

Test generation timeout is exceeded 2-3 times for every StringExamples.

To Reproduce

  1. Run the 'UTBotJava' project in IntelliJ Idea 2022.2.2
  2. Install RC plugin build
  3. Open test generation dialog for utbot-sample/src/main/java/org/utbot/examples/strings/StringExamples.java
  4. Select both classes: IntPair and StringExamples, timeout is 60 sec
  5. Check when test generation finishes.

Expected behavior

Tests are supposed to be generated during the defined timeout of 60 sec.

Actual behavior

Tests are generation for about 4 minutes 45 sec - up to 5 minutes

Environment

Windows 10 Pro
RAM 32.0 GB
Processor Intel(R) Core(TM) i7-9700K CPU @ 3.60GHz 3.60 GHz
IntelliJ IDEA 2022.1.4
Heap memory allocated for IDEA is 2 GB

Additional context

Originally posted by @alisevych in #1102 (comment)

@dtim has made some investigation on the Release 2022.10 build

Activity

korifey
added this to UTBot Javaon Oct 11, 2022
korifey
moved this to Todo in UTBot Javaon Oct 11, 2022
alisevych
mentioned this in 2 issues on Oct 11, 2022
alisevych
added
ctg-bugIssue is a bug
on Oct 11, 2022
alisevych

alisevych commented on Oct 11, 2022

@alisevych
alisevych
on Oct 11, 2022
MemberAuthor

probably related to other performance issues:
#1028
#1027

alisevych
assigned
Domonion
on Oct 11, 2022
dtim

dtim commented on Oct 12, 2022

@dtim
dtim
on Oct 12, 2022
Collaborator

I just tried to reproduce this issue locally (I used a debug IDE instead of the RC plugin because the linked RC plugin does not contain the fix for running child processes in Linux, so no tests can be generated at all). For me (Ubuntu 20.04.5 LTS, main branch with last commit 5af1f4888cbcf5d96ab935af37234b9dc0515962), test generations correctly stops in 60 seconds. Of course, most of tests are fuzzed, because the symbolic engine requires more time to process all methods). Method coverage is 100%, line coverage is 85%.

Domonion

Domonion commented on Oct 12, 2022

@Domonion
Domonion
on Oct 12, 2022 · edited by Domonion
Collaborator

I also tried to reproduce in on W10, 312e34795e18decee752e98cc2f3e3499e21194e, also 60sec

Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz   2.11 GHz
16.0 GB (15.8 GB usable)
dtim

dtim commented on Oct 12, 2022

@dtim
dtim
on Oct 12, 2022 · edited by dtim
Collaborator

With fuzzer turned off in settings.properties, test generation also stops in 60 seconds for me. I tried to generate tests in the empty test root, and not I am unable to actually run these tests, but it seems that it is related to some AWT issues under Linux in the debug Idea itself. I'll check it out one more time when I can run the plugin binary in my main Idea.

Hardware specs:

Intel(R) Core(TM) i7-10875H CPU @ 2.30GHz (8 cores)
Total online memory:    31,9G
dtim
mentioned this on Oct 12, 2022
Domonion

Domonion commented on Oct 12, 2022

@Domonion
Domonion
on Oct 12, 2022
Collaborator

Repro step-by-step:

  1. open UTBotJava project in Idea 2022.2.3
  2. install utbot plugin
  3. if utbot was installed - reload idea. Otherwise do not
  4. Generate tests for StringExamples.kt
Domonion

Domonion commented on Oct 12, 2022

@Domonion
Domonion
on Oct 12, 2022
Collaborator

Also following exception is generated

com.intellij.openapi.project.IndexNotReadyException: Please change caller according to com.intellij.openapi.project.IndexNotReadyException documentation
	at com.intellij.openapi.project.IndexNotReadyException.create(IndexNotReadyException.java:67)
	at com.intellij.util.indexing.FileBasedIndexImpl.handleDumbMode(FileBasedIndexImpl.java:844)
	at com.intellij.util.indexing.FileBasedIndexImpl.ensureUpToDate(FileBasedIndexImpl.java:792)
	at com.intellij.psi.stubs.StubIndexEx.getContainingIds(StubIndexEx.java:350)
	at com.intellij.psi.stubs.StubIndexEx.processElements(StubIndexEx.java:157)
	at com.intellij.psi.stubs.StubIndex.getElements(StubIndex.java:99)
	at com.intellij.psi.stubs.StubIndex.getElements(StubIndex.java:87)
	at com.intellij.psi.impl.java.stubs.index.JavaFullClassNameIndex.get(JavaFullClassNameIndex.java:34)
	at com.intellij.psi.impl.file.impl.JavaFileManagerImpl.doFindClasses(JavaFileManagerImpl.java:88)
	at com.intellij.psi.impl.file.impl.JavaFileManagerImpl.findClass(JavaFileManagerImpl.java:111)
	at com.intellij.psi.impl.PsiElementFinderImpl.findClass(PsiElementFinderImpl.java:40)
	at com.intellij.psi.impl.JavaPsiFacadeImpl.doFindClass(JavaPsiFacadeImpl.java:93)
	at com.intellij.psi.impl.JavaPsiFacadeImpl.findClass(JavaPsiFacadeImpl.java:71)
	at com.intellij.psi.impl.PsiClassImplUtil.findSpecialSuperClass(PsiClassImplUtil.java:748)
	at com.intellij.psi.impl.PsiClassImplUtil.getSuperClass(PsiClassImplUtil.java:740)
	at com.intellij.psi.impl.PsiClassImplUtil.getSupersInner(PsiClassImplUtil.java:792)
	at com.intellij.psi.impl.PsiClassImplUtil.getSupers(PsiClassImplUtil.java:752)
	at com.intellij.psi.impl.source.PsiClassImpl.getSupers(PsiClassImpl.java:215)
	at com.intellij.psi.impl.PsiClassImplUtil$MemberCache.lambda$new$1(PsiClassImplUtil.java:356)
	at com.intellij.util.containers.FilteredTraverserBase$Meta.childrenImpl(FilteredTraverserBase.java:437)
	at com.intellij.util.containers.FilteredTraverserBase$Meta.children(FilteredTraverserBase.java:423)
	at com.intellij.util.containers.FilteredTraverserBase.children(FilteredTraverserBase.java:304)
	at com.intellij.util.containers.TreeTraversal$2$1WrappedTree.fun(TreeTraversal.java:121)
	at com.intellij.util.containers.TreeTraversal$2$1WrappedTree.fun(TreeTraversal.java:108)
	at com.intellij.util.containers.TreeTraversal$P.iterable(TreeTraversal.java:805)
	at com.intellij.util.containers.TreeTraversal$P.iterator(TreeTraversal.java:799)
	at com.intellij.util.containers.TreeTraversal$BiOrderIt.nextImpl(TreeTraversal.java:540)
	at com.intellij.util.containers.JBIterator.peekNext(JBIterator.java:150)
	at com.intellij.util.containers.JBIterator.hasNext(JBIterator.java:110)
	at com.intellij.util.containers.ContainerUtil.newArrayList(ContainerUtil.java:133)
	at com.intellij.util.containers.JBIterable.toList(JBIterable.java:793)
	at com.intellij.util.containers.FilteredTraverserBase.toList(FilteredTraverserBase.java:309)
	at com.intellij.psi.impl.PsiClassImplUtil$MemberCache.<init>(PsiClassImplUtil.java:359)
	at com.intellij.psi.impl.PsiClassImplUtil.lambda$getMap$1(PsiClassImplUtil.java:194)
	at com.intellij.util.containers.ConcurrentFactoryMap$2.create(ConcurrentFactoryMap.java:174)
	at com.intellij.util.containers.ConcurrentFactoryMap.get(ConcurrentFactoryMap.java:40)
	at com.intellij.psi.impl.PsiClassImplUtil.getMap(PsiClassImplUtil.java:194)
	at com.intellij.psi.impl.PsiClassImplUtil.processCachedMembersByName(PsiClassImplUtil.java:532)
	at com.intellij.psi.impl.PsiClassImplUtil.processDeclarationsInClass(PsiClassImplUtil.java:444)
	at com.intellij.psi.impl.PsiClassImplUtil.processDeclarationsInClass(PsiClassImplUtil.java:421)
	at com.intellij.psi.impl.source.PsiClassImpl.processDeclarations(PsiClassImpl.java:479)
	at com.intellij.psi.scope.util.PsiScopesUtil.treeWalkUp(PsiScopesUtil.java:61)
	at com.intellij.psi.scope.util.PsiScopesUtil.treeWalkUp(PsiScopesUtil.java:40)
	at com.intellij.psi.scope.util.PsiScopesUtil.resolveAndWalk(PsiScopesUtil.java:207)
	at com.intellij.psi.scope.util.PsiScopesUtil.resolveAndWalk(PsiScopesUtil.java:136)
	at com.intellij.psi.impl.source.PsiJavaCodeReferenceElementImpl.resolve(PsiJavaCodeReferenceElementImpl.java:493)
	at com.intellij.psi.impl.source.PsiJavaCodeReferenceElementImpl.access$100(PsiJavaCodeReferenceElementImpl.java:44)
	at com.intellij.psi.impl.source.PsiJavaCodeReferenceElementImpl$OurGenericsResolver.resolve(PsiJavaCodeReferenceElementImpl.java:379)
	at com.intellij.psi.impl.source.PsiJavaCodeReferenceElementImpl$OurGenericsResolver.resolve(PsiJavaCodeReferenceElementImpl.java:372)
	at com.intellij.psi.impl.source.resolve.ResolveCache.lambda$resolveWithCaching$2(ResolveCache.java:184)
	at com.intellij.openapi.util.Computable.get(Computable.java:16)
	at com.intellij.psi.impl.source.resolve.ResolveCache.lambda$loggingResolver$4(ResolveCache.java:260)
	at com.intellij.openapi.util.RecursionManager$1.computePreventingRecursion(RecursionManager.java:114)
	at com.intellij.openapi.util.RecursionGuard.doPreventingRecursion(RecursionGuard.java:44)
	at com.intellij.openapi.util.RecursionManager.doPreventingRecursion(RecursionManager.java:68)
	at com.intellij.psi.impl.source.resolve.ResolveCache.resolve(ResolveCache.java:237)
	at com.intellij.psi.impl.source.resolve.ResolveCache.resolveWithCaching(ResolveCache.java:184)
	at com.intellij.psi.impl.PsiImplUtil.multiResolveImpl(PsiImplUtil.java:798)
	at com.intellij.psi.impl.source.PsiJavaCodeReferenceElementImpl.getCanonicalText(PsiJavaCodeReferenceElementImpl.java:294)
	at com.intellij.psi.impl.source.PsiJavaCodeReferenceElementImpl.getCanonicalText(PsiJavaCodeReferenceElementImpl.java:284)
	at com.intellij.psi.impl.source.PsiClassReferenceType.getText(PsiClassReferenceType.java:246)
	at com.intellij.psi.impl.source.PsiClassReferenceType.getCanonicalText(PsiClassReferenceType.java:233)
	at com.intellij.psi.PsiClassType$Stub.getCanonicalText(PsiClassType.java:352)
	at org.utbot.intellij.plugin.util.SignaturesHelperKt.signature(SignaturesHelper.kt:17)
	at org.utbot.intellij.plugin.util.SignaturesHelperKt.signature(SignaturesHelper.kt:13)
	at org.utbot.intellij.plugin.generator.UtTestsDialogProcessor.findMethodsInClassMatchingSelected(UtTestsDialogProcessor.kt:350)
	at org.utbot.intellij.plugin.generator.UtTestsDialogProcessor.access$findMethodsInClassMatchingSelected(UtTestsDialogProcessor.kt:67)
	at org.utbot.intellij.plugin.generator.UtTestsDialogProcessor$createTests$1$1$run$1$1.compute(UtTestsDialogProcessor.kt:177)
	at org.utbot.intellij.plugin.generator.UtTestsDialogProcessor$createTests$1$1$run$1$1.compute(UtTestsDialogProcessor.kt:175)
	at com.intellij.openapi.project.DumbService.lambda$runReadActionInSmartMode$0(DumbService.java:113)
	at com.intellij.openapi.project.DumbService.runReadActionInSmartMode(DumbService.java:144)
	at com.intellij.openapi.project.DumbService.runReadActionInSmartMode(DumbService.java:113)
	at org.utbot.intellij.plugin.generator.UtTestsDialogProcessor$createTests$1$1$run$1.call(UtTestsDialogProcessor.kt:175)
	at org.utbot.intellij.plugin.generator.UtTestsDialogProcessor$createTests$1$1$run$1.call(UtTestsDialogProcessor.kt:161)
	at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.insideReadAction(NonBlockingReadActionImpl.java:536)
	at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.lambda$attemptComputation$3(NonBlockingReadActionImpl.java:501)
	at com.intellij.openapi.application.impl.ApplicationImpl.tryRunReadAction(ApplicationImpl.java:1154)
	at com.intellij.openapi.progress.util.ProgressIndicatorUtils.lambda$runInReadActionWithWriteActionPriority$0(ProgressIndicatorUtils.java:75)
	at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runActionAndCancelBeforeWrite(ProgressIndicatorUtils.java:158)
	at com.intellij.openapi.progress.util.ProgressIndicatorUtils.lambda$runWithWriteActionPriority$1(ProgressIndicatorUtils.java:115)
	at com.intellij.openapi.progress.ProgressManager.lambda$runProcess$0(ProgressManager.java:66)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$2(CoreProgressManager.java:188)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$executeProcessUnderProgress$12(CoreProgressManager.java:608)
	at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:683)
	at com.intellij.openapi.progress.impl.CoreProgressManager.computeUnderProgress(CoreProgressManager.java:639)
	at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:607)
	at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:60)
	at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:175)
	at com.intellij.openapi.progress.ProgressManager.runProcess(ProgressManager.java:66)
	at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runWithWriteActionPriority(ProgressIndicatorUtils.java:112)
	at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runInReadActionWithWriteActionPriority(ProgressIndicatorUtils.java:75)
	at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.attemptComputation(NonBlockingReadActionImpl.java:501)
	at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.executeSynchronously(NonBlockingReadActionImpl.java:428)
	at com.intellij.openapi.application.impl.NonBlockingReadActionImpl.executeSynchronously(NonBlockingReadActionImpl.java:186)
	at org.utbot.intellij.plugin.generator.UtTestsDialogProcessor$createTests$1$1.run(UtTestsDialogProcessor.kt:180)
	at com.intellij.openapi.progress.impl.CoreProgressManager.startTask(CoreProgressManager.java:442)
	at com.intellij.openapi.progress.impl.ProgressManagerImpl.startTask(ProgressManagerImpl.java:114)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcessWithProgressAsynchronously$5(CoreProgressManager.java:493)
	at com.intellij.openapi.progress.impl.ProgressRunner.lambda$submit$3(ProgressRunner.java:252)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$2(CoreProgressManager.java:188)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$executeProcessUnderProgress$12(CoreProgressManager.java:608)
	at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:683)
	at com.intellij.openapi.progress.impl.CoreProgressManager.computeUnderProgress(CoreProgressManager.java:639)
	at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:607)
	at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:60)
	at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:175)
	at com.intellij.openapi.progress.impl.ProgressRunner.lambda$submit$4(ProgressRunner.java:252)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:702)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:699)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1.run(Executors.java:699)
	at java.base/java.lang.Thread.run(Thread.java:833)
	```
Domonion
mentioned this on Oct 13, 2022
Domonion
closed this as completedin #1151on Oct 14, 2022
Repository owner moved this from Todo to Done in UTBot Javaon Oct 14, 2022
alisevych
added
status-verifiedBug fix is verified
on Oct 17, 2022
alisevych

alisevych commented on Oct 17, 2022

@alisevych
alisevych
on Oct 17, 2022
MemberAuthor

Verified on the build from latest main - Not reproducing.

alisevych
mentioned this on Nov 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

ctg-bugIssue is a bugstatus-verifiedBug fix is verified

Type

No type

Projects

UTBot Java

Status

Done

Milestone

No milestone

Relationships

None yet

    Development

    Participants

    @dtim@Domonion@alisevych

    Issue actions
      StringExamples generation exceeds timeout in 2-3 times · Issue #1141 · UnitTestBot/UTBotJava