Question about Transfer performance and data routing

[deardooley]

I'm learning about onedata at the moment an I have a few questions about remote transfer performance.

• What protocol is currently used to move data between distributed oneproviders?
• Is that the same protocol used to move the data from a oneprovider to a oneclient?
• Is data proxying through an intermediate node once it leaves the oneprovider, or does it go point to point between oneproviders an oneclients?

[lopiola]

Sorry it took so long, but here are the answers:

• We use are own protocol called rtransfer - Oneproviders open multiple channels between each other and transfer the data in parallel (chunked). If configured properly, it is able to utilize most of the actual link throughput.
• No, Oneclient typically reads or writes data in small blocks and requires another protocol with different optimizations, called ProxyIO. Due to the round trip delays on each block it is not viable for HPC, you should use DirectIO instead (when the Oneclient has direct access to the storage systems and contacts Oneprovider only to find the file location and metadata).
• Currently, the data is proxied through the intermadiate Oneprovider. So, if you connect with a Oneclient to ProviderAlpha and ask for a file that is located in ProviderBeta, the request will be proxied and the data will be returned via ProviderAlpha. Our roadmap for the future includes the ability of Oneclient to switch betweeen Oneproviders in the background and get rid of proxy requests.

Closing the issue for now, let us know if you have any further questions.