Made ITs that restart MAC faster#5246
Merged
dlmarion merged 4 commits intoapache:2.1from Jan 15, 2025
Merged
Conversation
Modified MAC so that it cleaned up lock paths in ZooKeeper and ZooCache when stopping. Noticed that in tests that restarted MAC the Manager process would wait for the previous lock to be removed on the session timeout. The lock paths would also be cached in ZooCache and not updated right away because the Watcher would not fire when MAC was stopped, a ConnectionLoss error would be returned when MAC started, and it would take a while for ZooCache to fix itself.
Contributor
Author
|
These changes shaved about 30s off of VolumeIT locally for me in 2.1. In main, with more tests, it shaved off about 50s. |
keith-turner
approved these changes
Jan 11, 2025
ctubbsii
reviewed
Jan 12, 2025
core/src/main/java/org/apache/accumulo/core/fate/zookeeper/ZooCache.java
Outdated
Show resolved
Hide resolved
minicluster/src/main/java/org/apache/accumulo/miniclusterImpl/MiniAccumuloClusterControl.java
Outdated
Show resolved
Hide resolved
minicluster/src/main/java/org/apache/accumulo/miniclusterImpl/MiniAccumuloClusterImpl.java
Outdated
Show resolved
Hide resolved
minicluster/src/main/java/org/apache/accumulo/miniclusterImpl/MiniAccumuloClusterImpl.java
Show resolved
Hide resolved
ctubbsii
reviewed
Jan 14, 2025
minicluster/src/main/java/org/apache/accumulo/miniclusterImpl/MiniAccumuloClusterImpl.java
Outdated
Show resolved
Hide resolved
minicluster/src/main/java/org/apache/accumulo/miniclusterImpl/MiniAccumuloClusterImpl.java
Show resolved
Hide resolved
ctubbsii
reviewed
Jan 14, 2025
Member
ctubbsii
left a comment
ctubbsii
left a comment
There was a problem hiding this comment.
Overall, looks good. I made a few small suggestions about the implementation. The discussion about ZooCache I think is largely a red herring. The main benefit here is just to avoid waiting for those ephemeral lock nodes to go away, and that's true regardless of what ZooCache is doing. I think ZooCache is handling the disconnect event correctly either way.
minicluster/src/main/java/org/apache/accumulo/miniclusterImpl/MiniAccumuloClusterControl.java
Outdated
Show resolved
Hide resolved
minicluster/src/main/java/org/apache/accumulo/miniclusterImpl/MiniAccumuloClusterControl.java
Outdated
Show resolved
Hide resolved
minicluster/src/main/java/org/apache/accumulo/miniclusterImpl/MiniAccumuloClusterImpl.java
Outdated
Show resolved
Hide resolved
server/base/src/main/java/org/apache/accumulo/server/util/ZooZap.java
Outdated
Show resolved
Hide resolved
ctubbsii
approved these changes
Jan 15, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Modified MAC so that it cleaned up lock paths
in ZooKeeper and ZooCache when stopping. Noticed
that in tests that restarted MAC the Manager
process would wait for the previous lock to be
removed on the session timeout. The lock paths
would also be cached in ZooCache and not updated
right away because the Watcher would not fire
when MAC was stopped, a ConnectionLoss error
would be returned when MAC started, and it would
take a while for ZooCache to fix itself.