HDDS-5717. Refactor TestOzoneManagerListVolumes to reuse mini-ozone clusters#2615
Merged
sodonnel merged 3 commits intoapache:masterfrom Sep 15, 2021
Merged
HDDS-5717. Refactor TestOzoneManagerListVolumes to reuse mini-ozone clusters#2615sodonnel merged 3 commits intoapache:masterfrom
sodonnel merged 3 commits intoapache:masterfrom
Conversation
Contributor
Author
|
The last run (which failed for some other reason) ran in: Saving about 166 seconds. |
|
|
||
| this.allowListAllVolumes = conf.getBoolean(OZONE_OM_VOLUME_LISTALL_ALLOWED, | ||
| OZONE_OM_VOLUME_LISTALL_ALLOWED_DEFAULT); | ||
| setInstanceVariablesFromConf(); |
Contributor
There was a problem hiding this comment.
Minor: Similarly do we need to remove isAclEnabled from OzoneManager L440
Contributor
Author
There was a problem hiding this comment.
Ah, you are correct. Well spotted. I have fixed this and pushed another commit.
Contributor
bharatviswa504
left a comment
bharatviswa504
left a comment
There was a problem hiding this comment.
Overall LGTM, except one minor comment.
Contributor
Author
|
@bharatviswa504 Are you happy with the updated commit? Are you ok for me to commit this now? |
Contributor
Author
|
I am going to commit this now, as @bharatviswa504 gave a "LGTM" earlier pending one minor comment which I addressed. Let me know if you have any concerns and we can address them. |
Contributor
Author
|
I spoke to @bharatviswa504 over slack - he is travelling this week, but gave me his +1 "offline", so I am going ahead with the commit. |
errose28
added a commit
to errose28/ozone
that referenced
this pull request
Sep 30, 2021
* HDDS-4944: (81 commits) HDDS-5750. [Multi-Tenant] GetS3Secret should retrieve secret from new tables as well (apache#2649) HDDS-5476. [Multi-Tenant] Support Ozone s3 authentication with arbitrary accessId that is not same as the kerberos ID (apache#2635) HDDS-5770. Silent failures of k3s install are difficult to debug (apache#2667) HDDS-5759. Bump aspectj version (apache#2658) HDDS-5773. Avoid code duplication for mini cluster without datanodes (apache#2669) HDDS-5691. Restrict Recon NSSummaryEndpoint and ContainerEndpoint to admins. (apache#2638) HDDS-5771. Speed up TestDatanodeHddsVolumeFailureToleration by reducing dead interval (apache#2668) HDDS-5767. Unit check may timeout (apache#2664) HDDS-5765. Test cluster provider possibly returns null (apache#2663) HDDS-5768. Skip safemode check in TestOzoneManagerRocksDBLogging (apache#2665) HDDS-5766. Speed up some OM tests by skipping SCM safemode check (apache#2662) HDDS-5761. should not shutdown om when setting a bigger bucket quota than volume quota (apache#2659) HDDS-5758. Speed up TestKeyInputStream and TestChunkInputStream by combining some tests (apache#2656) HDDS-5607. remove container manager v1 code (apache#2525) HDDS-5753. Split parts of misc suite (apache#2654) HDDS-5751. Use Mini Cluster Provider to speed up TestHDDSUpgrade (apache#2650) HDDS-5728. ContainerBalancer should use remaining space to calculate utilization. (apache#2625) HDDS-5402 Support list node based on NodeOperationalState and NodeState options in printTopology CLI (apache#2645) HDDS-5749. Reuse mini-clusters in TestOzoneFsHAURLs (apache#2647) HDDS-5717. Refactor TestOzoneManagerListVolumes to reuse mini-ozone clusters (apache#2615) ...
sadanand48
pushed a commit
that referenced
this pull request
Oct 2, 2021
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
TestOzoneManagerListVolumes takes about 215 seconds to run, as it has several tests, and each tests creates a new mini-cluster:
We can refactor the test to use the same Mini-Cluster and just restart the OM component between each test, saving a lot of time.
Note that this requires a small change in the OzoneManagerImpl class too.
On my laptop the runtime went from 176 seconds -> 20 seconds.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-5717
How was this patch tested?
Existing tests and a changes to existing tests.