HDDS-10263. Intermittent failure in freon/metadata-generate robot test

[adoroszlai]

What changes were proposed in this pull request?

Fix:

NoSuchFileException: There are not enough files for testing you should use CREATE_FILE to create at least batch-size * threads = 1000

which happens because only n=100 items are created, but List test requires 1000 items (batch-size = 100, threads = 10).

Reduce threads to 1.

https://issues.apache.org/jira/browse/HDDS-10263

How was this patch tested?

Ran freon/metadata-generate.robot locally 10x:

Metadata-Generate :: Test freon ommg command                                  
==============================================================================
[Setup] Create Volume and Buckets                                     | PASS |
------------------------------------------------------------------------------
[Read] Bucket Information                                             | PASS |
------------------------------------------------------------------------------
[Create] File in FILE_SYSTEM_OPTIMIZED Bucket                         | PASS |
------------------------------------------------------------------------------
[Read] File in FILE_SYSTEM_OPTIMIZED Bucket                           | PASS |
------------------------------------------------------------------------------
[List] File Status in FILE_SYSTEM_OPTIMIZED Bucket                    | PASS |
------------------------------------------------------------------------------
[List] light File status in FILE_SYSTEM_OPTIMIZED Bucket              | PASS |
------------------------------------------------------------------------------
[Create] Key in OBJECT_STORE Bucket                                   | PASS |
------------------------------------------------------------------------------
[Read] Key in OBJECT_STORE Bucket                                     | PASS |
------------------------------------------------------------------------------
[List] Keys in OBJECT_STORE Bucket                                    | PASS |
------------------------------------------------------------------------------
[List] Light Keys in OBJECT_STORE Bucket                              | PASS |
------------------------------------------------------------------------------
[Get] Key Information in OBJECT_STORE Bucket                          | PASS |
------------------------------------------------------------------------------
Metadata-Generate :: Test freon ommg command                          | PASS |
11 tests, 11 passed, 0 failed

CI:
https://github.com/adoroszlai/ozone/actions/runs/7730252484

[fapifta]

In principal I am ok with the proposed fix, however I don't understand why the failure was intermittent, @adoroszlai can you elaborate why this test could pass when it passes?

[xichen01]

In principal I am ok with the proposed fix, however I don't understand why the failure was intermittent, @adoroszlai can you elaborate why this test could pass when it passes?

ozone freon ommg --operation LIST_STATUS -n 1 --volume ${VOLUME} --bucket ${BUCKET_FSO} --batch-size ${n}

@fapifta The -n 1 means that the LIST_STATUS will only be executed one time. The default thread count is 10. If the first thread (threadSeqId=0) executes the LIST_STATUS, the startKeyName will be 0. Consequently, the testing will execute successfully. If the other threads (threadSeqId>0) execute the LIST_STATUS, the testing will fail.

ozone/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/OmMetadataGenerator.java

Lines 400 to 404 in b02741f

	startKeyName = getPath(threadSeqId * batchSize);
	keyArgs = omKeyArgsBuilder.get().setKeyName("").build();
	getMetrics().timer(operation.name()).time(() -> {
	List<OzoneFileStatus> fileStatusList = ozoneManagerClient.listStatus(
	keyArgs, false, startKeyName, batchSize);

[adoroszlai]

Thanks @fapifta, @xichen01 for the review.

[fapifta]

Thank @xichen01 for the explanation!