HDDS-7066. Include number of tests requested in Freon output#3638
Merged
kerneltime merged 3 commits intoapache:masterfrom Aug 8, 2022
Merged
HDDS-7066. Include number of tests requested in Freon output#3638kerneltime merged 3 commits intoapache:masterfrom
kerneltime merged 3 commits intoapache:masterfrom
Conversation
…e the command parameters --prefix and --number-of-tests used for the command at the start of test execution and ii.) if an error occurs, on exit of command to output a message indicating the number-of-objects expected and number that succeeded.
kerneltime
approved these changes
Aug 1, 2022
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
To improve description of error when verify freon tests,
ockv, are executed with default number-of-objects expected greater than the number of objects generated.Currently, the default
number-of-testsparameter is set to 1000. Under the condition that a freon testgeneratesa number of objects less than the default and is followed by a verification freon test thatverifysthe objects without specifying the number-of-tests, it fails with an error that is not descriptive of the cause. This patch fixes this error by adding 2 items to the console output of freon tests that make the error more obvious to the user:1.) On start of freon command execution, in addition to outputting to the console the --prefix parameter used, to also output the --number-of-tests parameter used (default or user specified).
2.) Should the command fail, as part of the print summary output to the console, to add the number-of-tests objects expected and the number succeeded.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-7066
How was this patch tested?
Manually tested ockv command without --number-of-tests set when number of objects generated < default (1000 objects).
generate freon test, generate 10 keys with
ockgozone freon ockg --volume=vol1 --bucket=freon --threads=12 --replication-type=RATIS --factor=THREE --size=134217728 --number-of-tests=10 --prefix=52128MYTESTTotal execution time (sec): 35
Failures: 0
Successful executions: 10
followed by a verify, validate freon test, validate keys without specifying
--number-of-tests, using default of 1000 withockv:ozone freon ockv --volume=vol1 --bucket=freon --threads=9 --prefix=52128MYTEST2022-07-28 17:00:10,322 [main] INFO impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties
2022-07-28 17:00:10,393 [main] INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
2022-07-28 17:00:10,393 [main] INFO impl.MetricsSystemImpl: ozone-freon metrics system started
2022-07-28 17:00:10,493 [main] INFO freon.BaseFreonGenerator: Executing test with prefix 52128MYTEST and number-of-tests 1000
...
| 0/1000 Time: 0:00:062022-07-28 17:41:23,620 [pool-2-thread-8] ERROR freon.BaseFreonGenerator: Error on executing task 10
KEY_NOT_FOUND org.apache.hadoop.ozone.om.exceptions.OMException: Key:52128MYTEST/10 not found
at org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.handleError(OzoneManagerProtocolClientSideTranslatorPB.java:696)
...
99.9% <= 5260.18 milliseconds
Total execution time (sec): 9
Failures: 2
Successful executions: 10
Expected 1000 --number-of-tests objects!, successfully executed 10
CI workflow.