GEODE-9441: Reduce Nested func test thread count #6708
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Change `NestedFunctionExecutionDistributedTest` to create fewer threads. The test creates a number of threads that is a multiple of `MAX_FE_THREADS`. On a CI machine, the default value of `MAX_FE_THREADS` can be very high, causing this test to create thousands of threads. This can leave the system unable to create threads for other processes, which in turn can cause Gradle processes or other tests to fail or hang. This test now sets `MAX_FE_THREADS` to a lower value (100), which suffices for the purpose of the test, and results in greatly reduced consumption of threads. Co-authored-by: Dale Emery <demery@vmware.com>
demery-pivotal
requested review from
boglesby,
kirklund,
nabarunnag and
sabbey37
as code owners
boglesby
reviewed
Jul 20, 2021
| @@ -111,6 +116,7 @@ private int createServerCache() throws IOException { | |||
|
|
|||
| private void createClientCache(int port, int functionTimeout) { | |||
| System.setProperty(GEMFIRE_PREFIX + "CLIENT_FUNCTION_TIMEOUT", String.valueOf(functionTimeout)); | |||
| System.setProperty(MAX_FE_THREADS_PROPERTY, TEST_MAX_FE_THREADS); | |||
There was a problem hiding this comment.
Its not necessary to set this value in the client. The Function Execution thread pool is not created there.
There was a problem hiding this comment.
Currently the client does MAX_FE_THREADS * 2 executions of ParentFunction. If I don't set MAX_FE_THREADS in the client, I get a swarm of "Pool unexpected socket timed out on client connection" failures with that many functions. Should I directly reduce the number of functions it executes, rather than computing from MAX_FE_THREADS?
There was a problem hiding this comment.
Sorry, you're right. You should set it.
boglesby
approved these changes
Jul 20, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Change
NestedFunctionExecutionDistributedTestto create fewer threads.The test creates a number of threads that is a multiple of
MAX_FE_THREADS. On a CI machine, the default value ofMAX_FE_THREADScan be very high, causing this test to create thousands of threads. This
can leave the system unable to create threads for other processes, which
in turn can cause Gradle processes or other tests to fail or hang.
This test now sets
MAX_FE_THREADSto a lower value (100), whichsuffices for the purpose of the test, and results in greatly reduced
consumption of threads.
Co-authored-by: Dale Emery demery@vmware.com