[FileCopy based bootstrap][FileCopyHandler] Add FCHandler#3035
[FileCopy based bootstrap][FileCopyHandler] Add FCHandler#3035DevenAhluwalia merged 26 commits intomasterfrom
Conversation
…lecopy-handler
…lecopy-handler
…lecopy-handler
…n/ambry into dw-filecopy-handler
…lecopy-handler
…lecopy-handler
...-file-transfer/src/main/java/com/github/ambry/filetransfer/handler/StoreFileCopyHandler.java
Show resolved
Hide resolved
| /** | ||
| * Configuration for FileCopyHandler | ||
| */ | ||
| public class FileCopyHandlerConfig { |
There was a problem hiding this comment.
Should we merge it to File Copy Based Replication Config. Creating separate config classes. will require too many config object to be sent from upstream classes.
There was a problem hiding this comment.
Imo, FCHConfig makes sense to exist on its own considering that more configs will come in later once Throttling piece is in the picture.
Currently preferring to not merge.
| @Nonnull ConnectionPool connectionPool, | ||
| @Nonnull StoreManager storeManager, | ||
| @Nonnull ClusterMap clusterMap, | ||
| @Nonnull FileCopyHandlerConfig config) { |
There was a problem hiding this comment.
Can we use File Copy config instead.
There was a problem hiding this comment.
Same as above. Currently preferring to not merge FCHConfig with FCBasedReplicationConfig.
| * Shutdown the file copy handler. Perform clean up steps in case of a graceful shutdown. | ||
| */ | ||
| void shutdown() { | ||
| connectionPool.shutdown(); |
There was a problem hiding this comment.
Are we planning to create a seperate connection pool for File Copy. We should not close it in File Copy Handler if this is a shared connection pool.
There was a problem hiding this comment.
imo, there's stop() method for stopping FCH and not closing connPool while shutdown is essentially to be called when the AmbryProcess is shutting down.
...-file-transfer/src/main/java/com/github/ambry/filetransfer/handler/StoreFileCopyHandler.java
Show resolved
Hide resolved
ambry-file-transfer/src/main/java/com/github/ambry/filetransfer/FileCopyThread.java
Show resolved
Hide resolved
| public void testGetFileCopyGetMetaDataResponseExpectConnectionTimeout() throws Exception { | ||
| // Arrange: Mock ConnectionPoolTimeoutException | ||
| when(handler.getOperationRetryHandler().executeWithRetry(any(), eq(GetMetadataWorkflow.GET_METADATA_OPERATION_NAME))) | ||
| .thenThrow(new ConnectionPoolTimeoutException("Timeout")); |
There was a problem hiding this comment.
Do we need to override the default timeout set for Connection Pool for File Copy or make it configurable?
There was a problem hiding this comment.
Already defined in fileCopyHandlerConnectionTimeoutMs in FCHConfig. Currently set to 5 seconds.
We'll need to tweak it to bigger values depending on chunk size. Also, this may be dynamic connection-timeout value for different operations (GetMetadata is still 5s meanwhile GetChunkData is like 30s etc).
These changes'll come in later once we are testing different chunk sizes.
ambry-file-transfer/src/main/java/com/github/ambry/filetransfer/handler/FileCopyHandler.java
Show resolved
Hide resolved
ambry-file-transfer/src/main/java/com/github/ambry/filetransfer/FileCopyInfo.java
Outdated
Show resolved
Hide resolved
3 participants
Summary
BUG=AMBRY-13117
Adds support for FileCopyHandler that orchestrates GetMetadata and GetChunk Apis given a Replica and a target peer node.
Testing Done
Test suite WIP