[DO NOT REVIEW] Stress test pubsub without docker #6722
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…shing to get a CI run of all the tests
- Rename existing StripedExecutor to StripedCoordinator - Add StripedExecutorServiceJUnitTest - Move new StripedExecutor into better package
This commit changes Geode's build system to run test worker processes outside of Docker. It replaces the Dockerized test plugin with a new plugin that assigns each test worker process a distinct range of ports. BUILD SYSTEM Changed `multi-process-test.gradle` to mark relevant test tasks as isolated tests. The `IsolatedTestPlugin` applies itself to test tasks marked this way. DOCKERIZED TEST PLUGIN Removed the dockerized test plugin. SPECIFY PORT RANGES VIA SYSTEM PROPERTIES Added `AvailablePort.lowerBound` and `AvailablePort.upperBound` system properties to specify the range of non-membership ports considered by the "get random port" methods of `AvaiablePort` and `AvailablePortHelper`. Changed `AvailablePort` and `AvailablePortHelper` to use the membership port range specified by Geode's existing `membership-port-range` system property, if defined. Changed `ProcessManager`to forward the test worker JVM's port range system properties to each child VM it creates. ISOLATED TEST PLUGIN Added the `IsolatedTestPlugin`, which overrides Gradle's built-in test worker process launcher with a custom mechanism: - `PortRange`: Defines a range of ports. - `PortRangeContext`: Defines the membership and non-membership port ranges for a process. Configures each `ProcessBuilder` to set the port range system properties for the process being launched. - `CompletableProcess`: Wraps a regular Java `Process` to execute an action on termination. - `PortRangeProcessLauncher`: Assigns each test worker process a distinct port range context. Launches test worker processes. Wraps each test worker `Process` in a `CompletableProcess` that releases its port range context when the process terminates. SPECIFIC TESTS Changed several tests in `DistributedSystemDUnitTest` and `InternalDistributedSystemJUnitTest` to clear the port range system properties added by the `IsolatedTestPlugin`. These tests verify that the system applies Geode's default port ranges. For these tests to work, those system properties must not be defined. Disabled a `NetstatDUnitTest` test that executes `lsof`. When this test runs CI outside of a Docker container, in the presence of dozens of tests and potentially hundreds of Geode members, the output of `lsof` is enormous. The `NetstatFunction` attempts to collect all of the output in memory, which results in an out of memory exception. Co-authored-by: Dale Emery <demery@vmware.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Thank you for submitting a contribution to Apache Geode.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:
Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?
Has your PR been rebased against the latest commit within the target branch (typically
develop)?Is your initial contribution a single, squashed commit?
Does
gradlew buildrun cleanly?Have you written or updated unit tests to verify your changes?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
Note:
Please ensure that once the PR is submitted, check Concourse for build issues and
submit an update to your PR as soon as possible. If you need help, please send an
email to dev@geode.apache.org.