HDDS-8013. Freon S3 bucket creation test should use unique prefix#5282
Merged
adoroszlai merged 3 commits intoapache:masterfrom Sep 14, 2023
Merged
HDDS-8013. Freon S3 bucket creation test should use unique prefix#5282adoroszlai merged 3 commits intoapache:masterfrom
adoroszlai merged 3 commits intoapache:masterfrom
Conversation
kerneltime
approved these changes
Sep 12, 2023
Xushaohong
reviewed
Sep 13, 2023
| Run Freon S3BG | ||
| [Setup] Setup aws credentials | ||
| Freon S3BG No newline at end of file | ||
| Freon S3BG s3bg-${BUCKET} |
Contributor
There was a problem hiding this comment.
Hi @adoroszlai, not understanding here what is the extra arg s3bg-${BUCKET} used for.
Seems the s3bg -p ${prefix} appointed the fixed prefix which is s3bg
Contributor
Author
There was a problem hiding this comment.
Robot keyword arguments can be specified as positional or named. So s3bg-${BUCKET} is for the first parameter, prefix.
ozone-ec.xml
2907:<msg timestamp="20230912 19:41:21.443" level="INFO">Running command 'ozone freon s3bg -e http://s3g:9878 -t 10 -n 100 -p s3bg-erasure 2>&1'.</msg>
ozone-ha.xml
14246:<msg timestamp="20230912 19:53:20.319" level="INFO">Running command 'ozone freon s3bg -e http://s3g:9878 -t 10 -n 100 -p s3bg-bucket-ozone-test-2251842976 2>&1'.</msg>
ozone-s3-haproxy.xml
2921:<msg timestamp="20230912 19:55:39.308" level="INFO">Running command 'ozone freon s3bg -e http://s3g:9878 -t 10 -n 100 -p s3bg-bucket-ozone-test-7696293952 2>&1'.</msg>
ozonesecure-ha.xml
21939:<msg timestamp="20230912 20:04:05.210" level="INFO">Running command 'ozone freon s3bg -e http://s3g:9878 -t 10 -n 100 -p s3bg-encrypted 2>&1'.</msg>
ozonesecure-vault.xml
3364:<msg timestamp="20230912 19:45:28.638" level="INFO">Running command 'ozone freon s3bg -e http://s3g:9878 -t 10 -n 100 -p s3bg-bucket-ozone-test-9555341792 2>&1'.</msg>
Xushaohong
approved these changes
Sep 13, 2023
Contributor
Xushaohong
left a comment
Xushaohong
left a comment
There was a problem hiding this comment.
Thx for the explanation @adoroszlai. LGTM
Contributor
Author
|
Thanks @kerneltime, @Xushaohong for the review. |
errose28
added a commit
to errose28/ozone
that referenced
this pull request
Sep 14, 2023
* master: (55 commits) HDDS-9236. Fix snapdiff output for key modification (apache#5258) HDDS-8013. Freon S3 bucket creation test should use unique prefix (apache#5282) HDDS-9228. Poor S3G read performance (apache#5274) HDDS-8941. Disable flaky TestContainerBalancerTask#testDelayedStart HDDS-1159. Remove flaky tag from TestContainerStateManagerIntegration (apache#5291) HDDS-6077. Remove flaky tag from TestAddRemoveOzoneManager (apache#5290) HDDS-6610. Remove support for recursive volume list/delete using ozone fs command (apache#5264) HDDS-7752. GetS3SecretRequest API should not return secret if secret of user already exists (apache#4538) HDDS-9173. Invalidate snapshot cache once snapshot gets purged (apache#5248) HDDS-8920. Ozone is supporting unicode volume and bucket names, unintentionally (apache#5276) HDDS-9275. LegacyReplicationManager: Delete excess unhealthy with force=true (apache#5286) HDDS-9264. Execute EC acceptance test in secure environment (apache#5279) HDDS-9161. Recon Pipelines datanode columns search does not work (apache#5213) HDDS-9107. Reduce the granularity of Container locks for BlockDeletingService (apache#5149) HDDS-9270. Create a script to list all acceptance test splits (apache#5281) HDDS-9220. Let ContainerBalancerConfiguration#toString print more info (apache#5228) HDDS-9208. Add queue limit in ReplicationServer. (apache#5216) HDDS-9268. [Snapshot] Update list of snapshot apis to include lsDiff details in docs. (apache#5278) HDDS-9234. OM should shutdown immediately if certificate durations are invalid (apache#5243) HDDS-9136. Throw exception when rename fails during moveToTrash. (apache#5253) ...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
S3 smoketests are executed with various bucket types (normal, link, encrypted, EC). Freon bucket creation test should use a unique prefix for each, because using the same prefix for all tests prevents testing actual bucket creation.
Actually, Freon S3 bucket creation test is independent of the bucket type being used, so we should tag
s3/freon.robotas such.https://issues.apache.org/jira/browse/HDDS-8013
How was this patch tested?
CI:
https://github.com/adoroszlai/hadoop-ozone/actions/runs/4872190259