HDDS-8382. Upgrade Ratis to 2.5.1

[adoroszlai]

What changes were proposed in this pull request?

Upgrade Ratis to 2.5.1.

Fix for compilation error in tests due to interface change was provided by @szetszwo. When merging this, please keep the following detail in the commit message:

Co-authored-by: Tsz-Wo Nicholas Sze <szetszwo@apache.org>

https://issues.apache.org/jira/browse/HDDS-8382

How was this patch tested?

CI:
https://github.com/adoroszlai/hadoop-ozone/actions/runs/4891132333

[adoroszlai]

TestDeleteWithSlowFollower is flaky (HDDS-8430), but it seems to be failing more frequently with new Ratis (passed only 13 / 100 runs).

[adoroszlai]

With Ratis 2.5.1, block is deleted contrary to test expectations. This results in two kinds of failures, depending on timing:

ozone/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestDeleteWithSlowFollower.java

Lines 305 to 306 in 7f7ec23

	Assert.assertTrue(
	containerData.getNumPendingDeletionBlocks() > numPendingDeletionBlocks);

ozone/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestDeleteWithSlowFollower.java

Lines 307 to 317 in 7f7ec23

	// make sure the chunk was never deleted on the leader even though
	// deleteBlock handler is invoked
	try {
	for (ContainerProtos.ChunkInfo chunkInfo : blockData.getChunks()) {
	keyValueHandler.getChunkManager()
	.readChunk(container, blockID, ChunkInfo.getFromProtoBuf(chunkInfo),
	null);
	}
	} catch (IOException ioe) {
	Assert.fail("Exception should not be thrown.");
	}

When block deletion reaches the following condition:

ozone/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/statemachine/background/BlockDeletingService.java

Lines 223 to 232 in 7f7ec23

	if (minReplicatedIndex >= 0 && minReplicatedIndex < containerBCSID) {
	LOG.warn("Close Container log Index {} is not replicated across all"
	+ " the servers in the pipeline {} as the min replicated "
	+ "index is {}. Deletion is not allowed in this container "
	+ "yet.", containerBCSID,
	containerData.getOriginPipelineId(), minReplicatedIndex);
	return false;
	} else {
	return true;
	}

with Ratis 2.5.1 minReplicatedIndex is always -1 (I guess due to RATIS-1767, but could be wrong), so deletion is allowed.

I think the minReplicatedIndex >= 0 condition is wrong. If minReplicatedIndex == -1, then we should not allow deleting the blocks, so I removed that condition. Test passes locally. Pushed new commit, waiting for CI in fork.

[smengcl]

Is this intended? minReplicatedIndex could be -1, thus getting rid of the minReplicatedIndex >= 0 condition here changes the logic slightly. oh I should've read the comment above first: #4664 (comment)

cc @szetszwo

[szetszwo]

I agree that it is correct to remove minReplicatedIndex >= 0.

[smengcl]

lgtm. but I'd prefer another eye on this.

[szetszwo]

+1 the change looks good.

[szetszwo]

I agree that it is correct to remove minReplicatedIndex >= 0.

[adoroszlai]

Thanks @smengcl for the review, @szetszwo for helping out with the state machine tests, reviewing and merging this.