Skip to content

Conversation

@Markoutte
Copy link
Collaborator

@Markoutte Markoutte commented Jan 12, 2023

Description

Use non-blocking job.join instead of spinlock for waiting until subjob is done (fixes #1659). Also, do not force reloading soot, because it takes budget time as much as 1 initialization does.

Fixes #1659

Type of Change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Manual Scenario

Run the example from the issue. It generates tests with coverage > 50%.

<StatsForClass(com.google.common.primitives.Shorts)> :
	canceled by timeout = true
	#methods = 49, 
	#methods started symbolic exploration = 49
	#methods with at least one TC = 24
	#methods with exceptions = 0
	#generated TC = 175
	#total coverage = 505/891
	#fuzzed coverage = 505/891
	#concolic coverage = 0/891

Also, profiler's result is much clearer:

image

Test generation with only `forceSootReload = false` optimization

<StatsForClass(com.google.common.primitives.Shorts)> :
	canceled by timeout = true
	#methods = 49, 
	#methods started symbolic exploration = 49
	#methods with at least one TC = 20
	#methods with exceptions = 0
	#generated TC = 90
	#total coverage = 464/891
	#fuzzed coverage = 464/891
	#concolic coverage = 0/891

image

Test generation with only `job.join` optimization

<StatsForClass(com.google.common.primitives.Shorts)> :
	canceled by timeout = true
	#methods = 49, 
	#methods started symbolic exploration = 49
	#methods with at least one TC = 23
	#methods with exceptions = 0
	#generated TC = 165
	#total coverage = 487/891
	#fuzzed coverage = 487/891
	#concolic coverage = 0/891

image

Checklist:

  • The change followed the style guidelines of the UTBot project
  • Self-review of the code is passed
  • The change contains enough commentaries, particularly in hard-to-understand areas
  • New documentation is provided or existed one is altered
  • No new warnings
  • New tests have been added
  • All tests pass locally with my changes

@sergeypospelov sergeypospelov merged commit 345580e into main Jan 13, 2023
@sergeypospelov sergeypospelov deleted the pelevin/contest_optimizations branch January 13, 2023 08:47
@alisevych alisevych added ctg-enhancement New feature, improvement or change request comp-contest-estimator Issue is related to contest estimator spec-performance Performance-related issue labels Jan 13, 2023
@alisevych alisevych added this to the 2023.03 Release milestone Mar 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp-contest-estimator Issue is related to contest estimator ctg-enhancement New feature, improvement or change request spec-performance Performance-related issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Performance problem with contest estimator that fails to create any test with 120 sec time budget

4 participants