Skip to content

HDDS-7066. Include number of tests requested in Freon output#3638

Merged
kerneltime merged 3 commits intoapache:masterfrom
neils-dev:HDDS-7066
Aug 8, 2022
Merged

HDDS-7066. Include number of tests requested in Freon output#3638
kerneltime merged 3 commits intoapache:masterfrom
neils-dev:HDDS-7066

Conversation

@neils-dev
Copy link
Contributor

@neils-dev neils-dev commented Jul 28, 2022

What changes were proposed in this pull request?

To improve description of error when verify freon tests, ockv, are executed with default number-of-objects expected greater than the number of objects generated.

Currently, the default number-of-tests parameter is set to 1000. Under the condition that a freon test generates a number of objects less than the default and is followed by a verification freon test that verifys the objects without specifying the number-of-tests, it fails with an error that is not descriptive of the cause. This patch fixes this error by adding 2 items to the console output of freon tests that make the error more obvious to the user:

1.) On start of freon command execution, in addition to outputting to the console the --prefix parameter used, to also output the --number-of-tests parameter used (default or user specified).
2.) Should the command fail, as part of the print summary output to the console, to add the number-of-tests objects expected and the number succeeded.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-7066

How was this patch tested?

Manually tested ockv command without --number-of-tests set when number of objects generated < default (1000 objects).

generate freon test, generate 10 keys with ockg
ozone freon ockg --volume=vol1 --bucket=freon --threads=12 --replication-type=RATIS --factor=THREE --size=134217728 --number-of-tests=10 --prefix=52128MYTEST
Total execution time (sec): 35
Failures: 0
Successful executions: 10

followed by a verify, validate freon test, validate keys without specifying --number-of-tests, using default of 1000 with ockv:

ozone freon ockv --volume=vol1 --bucket=freon --threads=9 --prefix=52128MYTEST
2022-07-28 17:00:10,322 [main] INFO impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties
2022-07-28 17:00:10,393 [main] INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
2022-07-28 17:00:10,393 [main] INFO impl.MetricsSystemImpl: ozone-freon metrics system started
2022-07-28 17:00:10,493 [main] INFO freon.BaseFreonGenerator: Executing test with prefix 52128MYTEST and number-of-tests 1000
...
| 0/1000 Time: 0:00:062022-07-28 17:41:23,620 [pool-2-thread-8] ERROR freon.BaseFreonGenerator: Error on executing task 10
KEY_NOT_FOUND org.apache.hadoop.ozone.om.exceptions.OMException: Key:52128MYTEST/10 not found
at org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.handleError(OzoneManagerProtocolClientSideTranslatorPB.java:696)
...
99.9% <= 5260.18 milliseconds

Total execution time (sec): 9
Failures: 2
Successful executions: 10
Expected 1000 --number-of-tests objects!, successfully executed 10

CI workflow.

…e the command parameters --prefix and --number-of-tests used for the command at the start of test execution and ii.) if an error occurs, on exit of command to output a message indicating the number-of-objects expected and number that succeeded.
@adoroszlai adoroszlai changed the title HDDS-7066. Improve Freon test behavior on error when number-of-tests not set HDDS-7066. Improve Freon test output on error Aug 3, 2022
@adoroszlai adoroszlai changed the title HDDS-7066. Improve Freon test output on error HDDS-7066. Include number of tests requested in Freon output Aug 3, 2022
@kerneltime kerneltime merged commit 7d50d29 into apache:master Aug 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants